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I hope that people who research computation theory or compiler theory will 
enjoy this book. For an introduction to these subjects I suggest another 
volume, such as Elements of the Theory of Computation by Christos 
Papadimitriou. My motivation for writing this was to explain the theoretical 
grounding for my 2005 "Turing" processor architecture proposal, and thus 
justify its name. 


This is a work in progress, which is why I jokingly give it a version number. 
The original version was 0.1. The second major version added the C 
emulation code section, version .93. This document is version .94. It contains 
edits but little new material. There will be one or two more chapters added 
before we arrive at version 1.0. 


This document follows RFC 2119 for the definition of the words: may, should, 
and must. See https://www.rfc-editororg/rfc/rfc2119. Accordingly the 
word may signifies giving of permission or options. Consider this example. 
You are entering a subway station, and there is a turnstile that you must pass 
through to get inside. The turnstile is a state machine of sorts. With no coin 
inserted it is locked. With a coin inserted it allows one turn then goes back 
to the locked state. You look at the plaque by the coin slot and it says: "You 
may pass through the turnstile after depositing a coin." These directions are 
not making a prediction for the future, rather they are telling you what you 
are allowed to do. In this paper when I want to predict that something is 
probable to happen, but not certain, I will use the word might. This paper 
talks a lot about state machines, so understanding these definitions will be 
very helpful. 


Headings are first word capitalized. Technical terms or terms being defined 
are capitalized. For example, Turing Machine. Other words in headings are 
lower case. 
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Introduction 


Let us begin by contemplating the meaning of the term Turing Complete as it 
is applied to an instruction set architecture. 


An instruction set architecture document will describe a set of instructions 
and registers used at the assembly language level for programming a 
microprocessor. Among the instructions there will be some that either 
directly or through side effects load and store data from a system memory. 
Unless I/O is memory mapped, other instructions will be for moving data to 
and from peripheral channels. It will almost always be the case that there is 
at least one such channel for accessing a secondary storage device. On 
modern machines there will also be some sort of interrupt architecture 
whereby various events may be programmed to invoke instruction 
sequences without having to explicitly call or branch to them. 


Typically the instruction set architecture is not used directly by 
programmers. Instead there will be a collection of programs which simplify 
and abstract the architecture interface so that the work of programming the 
computer can be divided among programmers of various disciplines. 


What happens at the layer below the instruction set architecture depends on 
the specifics of the computer realization. At this lower layer every single bit 
has to be taken care of using transistors and logic gates. Each transistor or 
logic gate must be manually combined to create the desired functions. Then, 
once a logic circuit is tested and found to be logically correct, devices used 
for realizing those circuits must be drawn to size, given places on silicon, and 
lines must be drawn between them to ‘wire them up’ 


Hence the instruction set architecture is a nice vehicle for communicating 
just enough information to compiler writers that they may port their tools, 
while also not conveying a lot of extra information that would be difficult for 
a non-specialist in hardware design to understand, and in any case would be 
irrelevant. 


Given two sufficiently expressive instruction set architectures for machines, 
say A and B, it will be possible to write an emulator program using 
instruction set A, where this emulator can read and run programs written 
using instruction set B. Such an emulator might be very useful if a person 
does not have in their possession hardware for instruction set B. Indeed it 
might even be the case that instruction set B was never intended to ever be 
implemented in hardware. For example, when instruction set B is Java 
language, which by its very design was intended to be emulated. 


A Turing Machine may be thought of as a minimalist instruction set 
architecture. We will develop this view further on in this volume. The Turing 
Machine was not designed for running programs, but rather was designed to 
assist humans in writing proofs about what computers can do. It is hard to 


8/148 


imagine a scenario where a literal realization of a Turing Machine like 
machine would have any practical value. 


Alan Turing described this mathematical abstraction for capturing the 
essence of computing in his paper of 1937, “On Computable Numbers With 
an Application to the Entscheidungsproblem”. His objective in doing this was 
to embody the meaning of ‘algorithm’. He used this abstraction as a tool to 
answer a question posed by Hilbert in 1928 as to whether an algorithm 
exists that can generally decide if a given logical statement is provable. 
Turing showed that in so far as algorithms are defined in terms of 
programming his machine model, that no such algorithm for generally 
deciding if statements are provable can exist. This was an earthshaking 
result. (Though it was not the first result to show limitations for a system of 
logic. Gédel’s Completeness theorem was published in 1929.) 


Turing’s machine is truly simple. It consists of a tape, a read-write head, and 
a state machine controller. The state controller chooses its next state based 
on the value under the head, and then it may choose to write a new value to 
tape, or to take a step. The Turing Machine becomes a true computer when 
the state controller is designed to read a description of Turing Machine logic 
from tape and then to execute that. The controller description found on the 
tape is a program encoded as data. Such a ‘Universal Turing Machine’ can 
emulate any other Turing Machine. 


Simpler models for computation are possible, as examples: finite state 
machines by themselves, or a state controller combined with a stack instead 
of a tape. However, such simplifications come at the cost of reduced 
expressiveness. There are some problems that Turing Machines can compute 
solutions for, which these other machines cannot. Hence these other 
machines are not Turing Complete. 


More complex models also exist, as examples: machines with multiple heads, 
multidimensional tapes, or even those that employ multiple co-operating 
Turing Machines. Yet these more complex machines are not computationally 
more expressive. Any problem that these machines can compute solutions 
for, so can a Turing Machine. So it appears that the Turing Machine is as 
simple as we can make a computer without diminishing its functional 
expressiveness. 


The Turing Machine cannot be realized exactly as it has been defined, 
because it makes use of an infinite tape. But this is of little hindrance for 
engineers. They would brush off this detail and create an approximate 
realization that has a very large tape. Engineers often get away with 
replacing the infinite with the very large because in our physical universe, it 
takes along time for computers to use a lot of memory. When a memory error 
does occur, engineers just call it a ‘memory fault’. 


If we had such a machine, the first thing we would discover is that it is 
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tedious to program. The instruction set architecture gives us no 
preconceived data types, not even integers, nor any instructions that 
accomplish arithmetic tasks. Typically in order to keep our lives simple when 
we are faced with programming a Turing Machine we use unary 
representation for integers. Accordingly the numbers 1, 2, 3, would be 
something like ‘s’, ‘ss’, ‘sss’. We could say that ‘s’ represents our successor 
function from Peano arithmetic. We then may write a program library with 
some useful functions. For example, to increment an integer the program 
looks under the read-write head, if it sees an ‘s’, it steps right. If it doesn’t see 
an ‘s’ then it writes an ‘s’ and halts. In this manner the string of ‘s’ symbols 
gets longer by one. We might then write subroutines to perform all the 
instructions found in a more conventional instruction set architecture, and 
in this manner construct an emulator. 


When we try to write programs of any complexity with our new emulator, 
we run into a problem that is one level up from the arithmetic itself: How can 
we make a memory map of variables that hold variable width data? Say we 
wanted to compile a program and place three integer variables next to each 
other on the tape. Then we would discover that if one of the variables grew 
in width, it would walk into the memory that was allocated for its neighbor 
and clobber its value. 


Along comes our engineer, and he has a practical fix for this problem. He 
notes that if we instead use Arabic representation, numbers will grow very 
slowly while they are incremented. Hence, he concludes that it will be 
sufficient to replace the need for an infinite allocation space for each integer 
with a large fixed width one. We call such a large fixed width allocation for 
our basic integer type a word. In addition our engineer will ask, how high 
does a person need to count anyway? 


But what happens in the highly unlikely case an integer does get too big to fit 
in one word of allocation? One possibility is that we call such an unlikely 
event an ‘exception’ and then stop the program. This exception is called 
‘integer overflow’. Another possibility is to saturate at the maximum value. 
More recently it has been proposed that we instead substitute in a special bit 
pattern that will be interpreted to stand for infinity and then keep 
computing. Today most computers will simply wrap back to zero and keep 
going as though nothing happened. But computers often have important jobs 
to do. It is not unimaginable that such a computer would be controlling a 
rocket, and then the rocket loses control and blows up, or something even 
worse could happen. 


There is one more solution to the exceptional case of integer overflow which 
has recently become popular in interpreted languages, and that is to use 
bignums. Accordingly, when the integer gets too big, we either move it, or 
places its pieces into a data structure such as a linked list. Both of these 
solutions typically require that we have something called an ‘address’ that 
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can be used to locate our integer or integer pieces. 


In terms of a Turing Machine tape, an address is the offset from the first tape 
cell. We may generate addresses by counting the steps taken while a machine 
simply steps right repeatedly. Thus an address is an integer, and the bignum 
solution does not solve the integer overflow problem, it just displaces the 
problem from our bignum integers to our address integers. This is fine with 
engineers, because they have now moved from 32 bit addresses to 64 bit 
addresses. 


Since the beginning of computing we have lived with the simplification that 
integers fit in fixed width words, and consequently have also lived with 
otherwise perfectly good programs terminating with exceptions. The 
problem typically gets worse during the transition to a new technology node 
when performance reaches a new plateau not imagined as being possible 
just a short time before. We might be nearing such a technology node now 
by moving to quantum computing. 


However, we live in a finite universe, and address spaces grow in size 
exponentially as we add more bits to addresses. Is there a point where the 
engineer’s solution will finally be sufficient? There are approximately 27°° 
protons in the universe. Hence with a 265 bit address, we could give every 
proton an address. Surely no memory would ever be made that is that large. 
An architecture using 265 bit fixed width addresses would never signal an 
‘address space is exhausted’ fault while running real programs, because any 
real program would have had an ‘out of physical memory’ fault before that 
could happen. Arguments of analogous form were used to justify using 32 bit 
addresses as the end all solution, and they are now being used to justify 64 
bit or 96 bit addresses as the end all solution. 


One possible hitch is that OS’s have a tendency to reserve blocks of address 
space. It is conceivable that an OS designer would create an address space 
management algorithm that is so inefficient, that despite memory being 
available, the OS still would not be able to map it. Perhaps because, with so 
many address bits the temptation was too strong not to use some as a 
content addressable memory. With content addressable memories large 
address widths are used for sparsely addressing relatively small memories 
so as to achieve searches with performance on the order of hardware decode 
circuit times. As another possibility, we might soon have to deal with state 
explosions in quantum computers, so perhaps we really will need all those 
bits. 


Independent of those possible scenarios, there is a here and now problem 
with using lots of bits to safeguard the fixed width data assumption. Namely 
that large fixed width values are an inefficient way to allocate memory. 
Addresses might be large numbers, but they still tend to congregate around 
a base value, i.e. indexes are typically small. The typical integer in a program 
does not exceed 12 bits, but it will be allocated in 32 or 64 bit words. If we 
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did not waste this space, caches could have higher hit rates, we could afford 
to run more threads, and we would get more work done with a given amount 
of system memory. 


And we have not even touched on the issues of floating-point representation. 


All this raises a question, is the fixed width assumption really necessary for 
practical computing? In a large part, the purpose of this volume is to provide 
an answer to this question, and the answer is no. 


In computation theory we often use ‘pumping’ type proofs to show that a 
particular machine is not Turing Complete. The idea is that we show some 
problem that a Turing Machine can solve independent of the length of input, 
and then we prove that for any solution on the non-Turing Complete 
machine, there will be a given length of input where the machine cannot 
solve the problem. 


Take for example, suppose that our input is a series of the symbol ‘a’ followed 
by a series of the symbol ‘b’, and we want the machine to tell us whether or 
not the number of ‘a’ symbols is equal to the number of ‘b’ symbols by 
printing a ‘y’ or a ‘n’ on the tape. 


€ y empty string 


ab y same number of a’s and b’s 
aon fewer b’s 
bon fewer a’s 
aaaaabbbbb y same number of a’s and b’s 
aaaaabbbb n fewer b’s 
aaaaabbbbbb n fewer a’s 


This is a pretty easy problem to solve on a Turing Machine. We can move the 
head to the ends and trim off one symbol from each of the strings of ‘a’s and 
‘b’s. If we meet in the middle we write ‘y’, otherwise we write ‘n’. If we 
consider that each of the strings is a unary number, then these operations 
can be thought of as subtracting one for each number and expecting both 
numbers to become zero on the same iteration. 


Now suppose we instead have a finite state machine. We can have a series of 
states that stand for one ‘a’, two ‘a's, three ‘a’s, etc. Each state in the series of 
‘a’ states would have a ‘b’ arc leading into the series of ‘b’ states that regress 
back, two ‘b’s, one ‘b’ and if the string ends with us on the marked state we 
know the ‘a’s and ‘b’s strings were equal in length. The problem is that we 
will only have a fixed number of states in a finite state machine, so there will 
be a bound to how many ‘a’s and ‘b’s that can be counted. 


We can discover the limitation of the state machine solution by pumping the 
input. First we give it length one strings, then length two strings, etc. Atsome 
point the machine will attempt to go past the last ‘a’ state, and then it will 
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hang or give us the wrong answer. 


Now let’s consider our engineer’s solutions for the unbounded memory 
problem, the address space problem, and the problem of creating a memory 
map for variable width variables. All of those solutions will break as various 
aspects of computation are pumped. We will get a fault after using all the 
memory. If we are swapping pages back to the hard drive we will get a fault 
when we run out of address space. If we have integers and a program that 
makes them growas it runs longer as the input gets longer, we eventually see 
an exception or other type of fault that causes the program to give wrong 
answers. 


It is for this reason that I would say that today’s instruction set architectures 
are not Turing Complete. When they are used in the manner they were 
designed to be used we cannot arbitrarily pump input lengths without things 
breaking. l.e. using the add instruction to perform addition will lead to 
overflows, load instructions accept address operands of fixed width, and the 
address translation system for the VM will only be able to handle a fixed 
number of page translations. 


But can we use our instruction set architecture to write a Turing Machine 
emulator? If so, then the instruction set architecture is Turing Complete, as 
the emulator can do anything that a Turing Machine can do. Theorists put no 
weight on a metric such as ‘what it was designed to do’. 


Any Turing Machine emulator approach must address how we are going to 
deal with the infinite tape. A simple approach is to divorce the question of 
the processor for being Turing Complete from that of the length of the tape. 
Say we use an actual magnetic tape transport as the secondary storage, and 
then say it is not the processor’s fault if we hit the end of a very long tape. 
Then all we have to do with the instruction set architecture is emulate a state 
controller. That is not a very high bar. However, still, with real tape storage 
we will run into faults when pumping that are not related to hitting the end 
of the tape. Tape storage mechanism use formatted tapes, and tape 
controllers support seek commands that accept address operands. At this 
point we can wave our hands and imagine a special tape drive and controller. 
One that has been designed for our purposes. This simplified device only 
accepts step right, step left, and read and write operations. It is OK for us to 
wave our hands and do this because it would be possible to build such a tape 
drive. Now our instruction set architecture truly only needs to emulate a 
state machine. 


This setup is nice in that the only fault will be that of physically running out 
of tape. However, this is also a pretty lame proof. It is hard to imagine an 
instruction set architecture that cannot emulate a finite state machine. In 
addition most all of the features of the given instruction set architecture will 
not be used. The message that an instruction set architecture is Turing 
Complete as determined using this set of assumptions carries little or no 
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interesting information. 


The remainder of this volume is about an instruction set architecture where 
the instructions in that architecture may be used directly in computation and 
the only faults we will have are due to program design, the properties of 
mathematics, or due to real resource limitations. This is my Turing Complete 
architecture. One might say that it is unique in that it is Turing Complete due 
to the features of the instruction set architecture itself, rather than in spite 
of them. 
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The TTCA Turing Machine variation 


avb 


command 


Figure 1 A Turing Machine 


Definition 


Alan Turing used a 5 tuple to define the Turing Machine, but his goals were 
different than ours. Our goal is to construct an architecture that does not 
have faults or exceptions as intrinsic properties. We want all exceptions to 
not be the caused by limitations of the architecture, but only be due to the 
mathematical realities, logical structure of the program, or real resource 
limitations. In a sense we are working in the opposite direction from which 
the Turing Machine was intended to be used. Rather than answering 
theoretical questions, we are trying to understand how to realize machines 
with good properties. Hence, I provide a more machine like, though 
equivalent, definition for a Turing Machine. A little further along in this text 
we will then modify this definition again. In this volume we will say that a 
Turing Machine is a set of the following elements: 


m 


fixed finite alphabet 

fixed distinct empty-symbol 
variable single ended tape 
variable read/write head 
variable read value buffer 
fixed state machine controller 
fixed left from leftmost error 
fixed start-state 

. fixed halt-state 

10. variable current-state 

11. fixed procedure for using these. 


OOND WN 


The values assigned to the variables are not part of the definition. 


A symbol may be any mathematical object for which there may be multiple 
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instances, where, within a given context, instances compare to be equal 
among other instances of the same symbol, but compare not equal to 
instances of other symbols. As a matter of convenience in this document, I 
use sequences of letters and/or digits for symbol instances. 


The alphabet is a finite set of symbols. 


The distinct empty-symbol may be any symbol that is not found in the 
alphabet. Like alphabet symbols, this symbol may appear in the tape 
sequence. 


Only instances of alphabet symbols or the empty-symbol may be written to 
the tape. Intuitively we might consider that the alphabet symbols are useful 
while the empty symbol is just taking up space while waiting to be displaced, 
in the same manner that we consider a bookshelf to be empty rather than 
being full of air. (And if we put a bookshelf underwater, is it still empty, or is 
it full of water?). 


The tape is an infinite sequence. We say that the first element in the tape 
sequence is the leftmost element. The second element in the sequence is then 
the right neighbor of the leftmost element. Etc. this nomenclature is drawn 
from a left to right writing convention for sequences.! The leftmost element 
has no left neighbor, but it does have a right neighbor. All other sequence 
elements have both a left neighbor and a right neighbor. A Turing Machine 
tape has no rightmost element because it is an infinite sequence. 


We notice three special subsequences: 


1. a first, leftmost, element which is either an alphabet symbol or the 
empty-symbol 

2. optionally this is followed by a sequence extending to a rightmost 
alphabet symbol 

3. if the second subsequence is finite, it is followed by an infinite 
sequence of empty-symbols. 


So as to distinguish this tape partitioning from others found in this volume, 
we give it the name, the 'Leftmost Rightmost Partitioning’, because it 
emphasizes the location of the leftmost element (alphabet or empty) and the 
rightmost alphabet element. 


It is because of this third subsequence, the infinite tail, that the empty- 
symbol became distinct from the alphabet. There is no way for a Turing 
Machine to compute an infinite empty tail, so such a tail must come from 


‘Later we will add addresses, and the tape then will have a number line under it, where 
the leftmost cell has a zero under it, the right neighbor has a one under it, etc. For 
writing systems that progress from right to left, or even up and down, we will want 
to maintain a convention that keeps the addresses increasing in the positive 
direction for a number line. 
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somewhere else. Here, we provide it by definition. 


Putting together the first two subsequences noted above gives us the active 
area. If there is no alphabet character on the tape, then there is no active area 
(i.e. the active area has length 0). The infinite right going tail of cells holding 
the empty-symbol is then the inactive area. 


The initial active area provided to a Turing Machine is called the input. The 
final active area, obtained after the Turing Machine halts, is called the output. 
If the input has been computed solely from another Turing Machine, then 
that input, will necessarily be finite in length. If the input is provided as a 
general mathematical object, perhaps even one abstracted from 'what a 
Turing Machine computation would produce if it ran to an infinite limit 
number of steps’, then the input may be either finite or infinite. 


The read/write head indicates? one of the elements in the tape sequence. The 
head is not a Natural Number as you are probably accustomed to for 
sequence element indexes. Such a definition would be circular because later 
we use a Turing Machine to define Natural Numbers and the concept of an 
address. Instead, we can think of the head as partitioning the tape into two 
subsequences. We will call this the 'Head Centric’ partitioning. The first 
subsequence extends from the leftmost element up to the indicated element, 
inclusively. During computation this first subsequence is necessarily finite. 
We call it the left-hand side. The second subsequence holds the rest of the 
tape. We call it the right-hand side. 


To step the head to the right we remove the leftmost element from the right- 
hand side, and then append it to, i.e. make it the new rightmost element of, 
the left-hand side. To step the head left, we remove the rightmost element of 
the left-hand side, and prepend it, i.e. make it the leftmost element of, the 
right-hand side. In the section, Addresses and Cells we will justify using the 
term cell in place of element, and then we will be able to say such things as 
the cell that the head is on, or the indicated cell. 


The controlling state machine is a graph. It may be either a Mealy or Moore 
style state machine graph, as these are equally expressive. Only the fixed 
procedure would change. We describe a Moore style machine and 
corresponding procedure here. The graph is a set containing three elements: 


1. set of states 
2. set of <state, command> pairings 
3. set of arcs. 


A state is a symbol. Instances of state symbols appear in a different context 
than those of alphabet symbols or the empty-symbol, and thus they do not 


Computer people have a habit of using the word 'indexes' in place of the word 
‘indicates’. Accordingly, they might say that an index variable indexes a value in an 
array, instead of saying that the index variable indicates a value in an array. 


17/148 


need to be distinct from them. 


Each state has a command attached to it, <state, command>. Each state 
appears in exactly one such pairing. We have a fixed set of commands: {do- 
nothing, step-left, step-right, and write(va/ue)}, where the va/ue 
parameter is set when the controller is defined. va/ue must be an instance 
of an alphabet symbol or an instance of the empty-symbol. In addition, we 
should not forget the implied read command that returns a value from the 
tape, and is used by the fixed procedure. 


In the section, More about commands, we discuss notational conventions for 
simplifying the expression of commands. 


The arcs are triples of symbols, <from-state, to-state, value>. The from-states, 
to-states, and values are set when the controller is defined. The value is an 
instance of either an alphabet symbol or the empty-symbol. We require that 
for each from-state that there exists exactly one arc for each member of the 
alphabet, one for the empty-symbol, and for states with a step-left 
command, that there be one arc for the left from leftmost error. Arcs can be 
dropped from diagrams if we can prove that they cannot be used. As a 
notational convenience we might also reduce the work of listing so many 
arcs by adopting the convention of explicitly providing only unique arcs, and 
then saying the rest of the arcs are identical to a default arc or an error arc. 


Here we present the fixed procedure in the form of a main procedure and 
two subprocedures, ‘Initialize’, and 'Take a Step’. 


Deterministic (Uniplex) Procedure 


Initialize 
1. place the input on the tape 
2. place the head on the leftmost element 
3. set the current state to the start state. 


Take a Step 
1. read the symbol instance indicated by the head 
2. execute the command corresponding to the current state 
3. select the arc that has a from-state that matches the 
current-state, and a value that matches the instance just read. 
Take that arc's next state and make it the machine's next 
state. 
4. make the next state the current state. 


Main 
1. Initialize 
2. if the current-state is the halt state, then halt 
3. Take a Step 
4. go to 2. 
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To perform a computation: follow the steps in Main, and its subprocedures, 
until being told to halt. 


We initialize the machine by instantiating the input on the tape, putting the 
head on the leftmost element of the tape, and setting the current state to the 
start state. 


We take a step by first reading the symbol under the head and storing that 
value in our read value buffer, executing the command associated with the 
state, and then following the arc selected by the element we read. We must 
read the element before executing the command, because the command 
might perform a write to the cell. Consequently, as part of the Turing Machine 
definition we picked up the buffer for holding the read value. 


Walking through this procedure is variously known as running the machine, 
performing a computation, evaluating the input, evaluating the function, or 
simply as computing. 


Note that while walking through this procedure we are always in exactly one 
state. This is because we required that there be one arc for each letter in the 
alphabet, empty-symbol, and possibly the left from leftmost error. We say 
that this property means that the machine is deterministic. 


To create a non-deterministic machine we allow multiple arcs to have the 
same read value as a second member, and we modify the second step of the 
procedure to follow all arcs that have matches with the symbol returned by 
read. When we do this we might find that zero, one, or many of the arcs are 
followed in one step, and thus that we may have zero or more next states. 
Conceptually each of these states corresponds to a separate machine head 
with its own read buffer. At the end of the step we rename our next state set 
to be the current state set. Should the halt state ever be placed in the current 
state set, then we halt the machine. Should the current state set become 
empty without every having reached a halt state, we say the machine has 
failed. 


Non-Deterministic (Multiplex) Procedure 


Initialize 
1. place the input on the tape 
2. for each start state place a <start state, head location on 
leftmost> pair in the current state set. 


Take a Step 
1. For each <state, head-location> member of the current 
state set: 
1.1. read the symbol instance from the 
corresponding head location. 
1.2. execute the command corresponding to the state. 
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1.3. for each arc match, add a <next-state, head- 
location> pair into the next state set. 
2. make the next state set the current state set. 


Main 
1. Initialize 
2. if the halt-state is a member of the current-state set, then 
halt, if current state set is empty, then halt with an error. 
3. Take a Step 
4. go to 2. 


As reviewed by Papadimitriou, such non-deterministic machines can be 
converted to deterministic ones. This is done by calling each unique ‘state 
set’ a state. You might recall from our definition of symbol, that we can use 
any mathematical object for which instances are comparable. That includes 
sets. When two current state sets have the same members, we say they are 
instances of the same state. When they have different members we say they 
are different states. Hence existence proofs on non-deterministic machines 
will turn out the same as for deterministic ones, but the non-deterministic 
machines might require fewer steps to evaluate. 


If we keep a trace of execution through a non-deterministic machine, and 
then walk backwards from each halt state, we find paths through the state 
machine, that, had we taken one of these paths, the machine would have 
reached the halt state. Had we known such a path in advance we could have 
saved ourselves from doing the work of following the other paths. This is why 
such machines are called 'non-deterministic' - we don't know which of the 
paths we are following while running forward will be the path that leads to 
the halt state first. We might arbitrarily pick a path, then we would have some 
probability of eventually hitting the halt state. 


But notice, that the conversion algorithm going from a non-deterministic 
machine to a deterministic machine is forward moving. It starts by giving the 
current state set a state name, and moves forward giving each unique set a 
state name, until hitting the halt state. No stochastic variables, or “guesses” 
are involved. Note also, when we run a non-deterministic machine we do so 
in a deterministic manner. There are no stochastic variables involved. 


Hence, what is non-deterministic is an interpretation rather than the 
machine definition or anything we are asked to do while walking through the 
fixed procedure. So as to avoid any confusion due to this nuance, I am going 
to introduce a different term for non-deterministic machines. I am going to 
say they are multiplexed. Should the need arise to emphasize that a machine 
is not multiplexed, I will say such a machine is unixplex. This new 
nomenclature is especially helpful when discussing the TM Library code, 
where the multiplexing of heads is readily apparent. 
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Levels of analysis 


When discussing systems of logic, we speak of first order logic, second order 
logic, etc. In complexity analysis we speak of the order of complexity of a 
program. With our Turing Machine descriptions we speak a great deal about 
the order of events, the order of elements on the tape, etc. It does not help 
the situation, that order also has multiple meanings as a colloquial word in 
the English language. One can give an order in the military, put one's office 
in order, or order the tasks in one's daily planner. So it is not surprising that 
in computer science the term meta has come into usage to speak about 
second order information. When meta is defined it is almost always 
described as second level information. People will even use both terms 
together, and say that information is 'meta level’. Hence, I will reduce the 
ambiguity associated with the term order within this documenta little bit by 
using the term level when speaking about the order of logic. This is not a 
small step for my document here, as second order logic plays a big role in the 
derivation of the TM Library, and I am concerned that by making this little 
terminology shift that I might lose some of the purists. 


We say that Turing Machines that halt in finite number of steps for any finite 
input are computational. l.e. for any given non-computational Turing 
Machine there will exist at least one input for which computation will never 
complete. By analyzing a machine we might learn that some machine, say M, 
is conditionally computational on a given set of inputs, say I. Then in case of 
the strongest delineation, computation will not halt for members of I, or it 
might be the case that we don't know about the inputs in I, or that I analysis 
shows that I can be broken into two sets, inputs for which computation will 
not stop for, and those for which analysis does not provide an answer. 


To analyze a given first Turing Machine we do not execute it, but instead we 
place its definition on the tape as input for a second Turing Machine. We then 
run this second Turing Machine so as to learn about the first one. (It is often 
the case that this second Turing Machine which performs the analysis is ina 
person's brain. Perhaps the person is writing a proof about the Turing 
Machine that is being studied.) Analysis can be applied to both 
computational and non-computational machines. In many cases it is possible 
to learn something useful about a non-computational machine, one which 
we cannot evaluate, by analyzing it. 


The term ‘analysis’ can be applied more generally. Accordingly, ‘first level’ 
analysis is the same as computation. ‘Second level’ analysis is what was 
described in the prior paragraph. When the term ‘analysis’ is used without 
further qualification, the level is implied by context, but typically we are 
referring to second level analysis. It is possible to perform higher levels of 
analysis. Here is a question that belongs to the next level: does there exist a 
Turing Machine, A, that can analyze a Turing Machine, B, to decide if B will 
always halt? We are standing above looking down, and asking a question 
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about machines of the second level. 
Addresses and Cells 


Natural Number, Address 


We can define a Turing Machine that is identical to the recursive definition 
of Natural Numbers as given by Peano. As the Natural Numbers never end, 
we cannot run this machine to a halting point, but we can analyze it. As such 
we can assign a Natural Number to each member of the tape sequence. We 
will call the machine that does this the address machine. 


Since multiple tape Turing Machines are equivalent to a single tape Turing 
Machines, we begin with the simplification and say that our address machine 
has two tapes. One we will call the main tape. We will assign addresses to the 
elements on the main tape. The second tape we will call the address tape. It 
is initially empty. 


The controller makes use of our increment routine for unary numbers which 
we described earlier. We begin with the main tape head on the leftmost 
element. The address tape reads zero, this is the address for the leftmost 
element. The controller then steps the main tape head right one element, and 
it runs the increment routine on the address tape. The address tape then 
reads ‘s’, i.e. one. This is the address for the right neighbor of the leftmost 
element. We then repeat this procedure while assigning addresses to each 
successive element, two, three, four, etc. We call the set of addresses created 
in this manner the ‘Natural Numbers’. 


Though we can never run this machine to the point where it halts, we may 
analyze the Natural Number machine and reason about it. As one example, 
we can take a first given Turing Machine with its head somewhere on a tape, 
encode that machine and its state, and also encode an address machine, and 
provide these two machines to a third machine, one called address-of: The 
address-of machine can then run the address machine while stepping the 
head left on the first given machine. At the point where the next left step 
would cause a left of leftmost error, the address will appear on the address 
tape of the address machine. 


Distance 

The distance between two elements is defined to be the difference in their 
corresponding addresses. 

Area and Length 


An area is a subsequence of a tape. An area may be defined using two 
addresses. The first address being that of the leftmost cell in the area, and 
the second address being that of the rightmost cell in the area. 
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The length of an area is one greater than the distance between the leftmost 
cell of the area, and that of the rightmost cell of the area. Hence the smallest 
length that an area may have is 1. This occurs when the address to the 
leftmost cell is the same as the address to the rightmost cell, i.e. when there 
is only one cell. 


Zero Length is a second level concept 


Suppose we have an area definition Turing Machine that is designed to mark 
an area based on some property of the symbols in that area. There are 
multiple possible definitions for such a Turing Machine, but let’s take a very 
simple one. Our area marking Turing Machine will step right until it finds the 
leftmost element of the area, and then write a marker symbol. It will then 
continue to step right and write a marker symbol until it reaches the 
rightmost element, which it will also mark. Then stepping and not finding 
the property, the machine will halt. 


Once an area is marked, we can go back and run a length measuring machine 
that counts the marks. 


Now suppose we employ a second level analysis. Instead of running our area 
marker Turing Machine, we examine its definition to learn if such a machine 
will halt. Although we know that it is not possible in general to analyze 
machines to know if they will halt, or not, this is certainly possible in some 
cases, and this is one of those cases. Upon analysis of our area marking 
machine we make a startling discovery - inputs exist for which the area 
marker will never halt. Namely, inputs which do not have a leftmost symbol 
with the property that defines the area we are looking for. 


We may say that inputs that have no leftmost symbol correspond to an area 
of zero length. We say this an abstract concept, because our area marking and 
area measuring machines will never be run then halt and return a length 
measure of zero. Instead we arrived at this conclusion that a non-existent 
area has zero length through inductive reasoning: Say we have an area of 
length i, and then remove one element from the area, then it has length i-1. 
As we repeat this, then eventually we will have an area of length 1 as 
discovered by our length assigning machine. Now we remove 1 more 
element. Each time we removed an element before, it made the length 
smaller by 1, so we reason that 1-1 = 0. The area is now length zero. We 
cannot go any further because there are no more elements to be removed. 


Note, then, that zero length is a second level concept. We cannot mark nor 
measure it, and given our first level definition for an area, nor can we even 
represent such an area at a first level. (For a second level definition of area, 
see the section Address of an Area.) 


This insight explains a lot of the pain related to the processing of end cases 
in computing. It will come up again in this volume when we talk about 
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emptiness and existence of our tape machine container, see the section 
Emptiness. It also explains why loops so often need to be primed or given 
special case guards, i.e. a layer of analysis. We will discuss this further later 
when introducing the first-rest pattern see Initialization and the First-Rest 
Pattern. 


It is interesting that non-existence has collided with zero length. This seems 
to be a contradiction, as something that does not exist should not have any 
length at all. All of this happens at the second level, at the level of analysis. At 
this level we can make a distinction between an area that we have given a 
name to, and perhaps a location, as compared to an area for which we have 
done neither of these things. Thus for purposes of second level analysis we 
will say that an area exists if it has a name or a location, even if it has zero 
length. We will say an area does not exist if it has neither a name nor a 
location. Again, execution of our first level area marking and length machines 
cannot provide us with any such information. 


Need for the concept of Cell 


Let us ask a question, what is it that an address is actually locating? Let us 
consider this question in the light of an example. Suppose we have the tape 
sequence of: 


a, b, C, 6,6, €... 


Now consider that we have an address of '2'. If we read tape address 2 we 
get back the letter 'c'. So the address is locating the 'c'. Now suppose we write 
at address 2. Say we write 'y'. Now our sequence is: 


a, b, Y, £, €, € .. 
Now we write 'I", resulting in the sequence: 
a, b, D, £, £, €... 


It would appear that the answer to our question is that address 2 is locating 
different things at different times. First it was, c, then y, then T. Yet the 
address did not change. It feels a little unsatisfactory to suggest that our 
concept of location depends on the value addressed. Also, notice, that when 
we created the Natural Number Turing Machine, that the values on the tape 
that was placed into correspondence with the Natural Numbers were never 
mentioned. Yet, we can't seem to answer the question of 'what is being 
addressed' without giving a value. 


Addresses speak to the structure of the tape rather than the values held on 
the tape. So as to facilitate this interpretation, we note that a sequence 
consists of a sequence of cells holding elements, rather than being just a 
sequence of elements. Given the concept of a cell we can say that an address 
always locates the same cell, though the contents of that cell can change. 
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This is only a small extension to the already existing concept of a variable in 
mathematics. In mathematics we allow that a variable can take on different 
values, though its name never changes. Now we are going to say that a cell 
can take on different values, though its address never changes. Furthermore, 
as the cellis part of the sequence, we are going to say the cell itself has a left 
neighbor or right neighbor - not the value in the cell. 


Address of an Area 


The address of a cell is the number of steps required to reach the cell when 
starting from the leftmost cell on the tape. The leftmost cell has an address 
of zero. It might seem intuitive to set the address of an area on the tape to be 
that of the first cell in the area. If we take a first level interpretation, and say 
that an area only exists if it has at least one cell, then this approach works 
well. However, what if we take a second level view of this question and say 
that the area exists but it has zero length? Then there is no first cell, so what 
is the location? 


Suppose that we are deleting the cells in an area.2 The delete command 
affects the cell to the right of the cell the head is on.* Say that we delete an 
area of three cells. We call delete three times. Now the area no longer exists, 
and it has length zero. However, the head never moved, so the left neighbor 
cell for an area is an invariant and we can use it for locating an area, even 
when the area has zero length. 


The inverse case also suggests that the cell to the left of an area defines its 
address for second level analysis. I.e. if we call append to grow an area, it 
grows to the right of the cell the head is on. This is important for our 
expanding tape variation. Before the first call to append, we have an address 
for the cell the head is on, but the area to the right does not exist yet. 


In a sense what we are doing with these delete and append examples is 
performing the discrete analog of a limit. This limit says that the address of 
an area is that of the left neighbor of the leftmost cell in the area. 


Now suppose we have two adjacent areas. 


3For finite tape up to the rightmost alphabet symbol we can emulate the deletion of cells 
on the conventional Turing Machine by moving all the elements on the right-hand 
side to the left. 

4 The reason for this is that we can't delete the cell that the head resides on as that would 
break the machine. I chose this convention over other possibilities that required 
moving the head, as those potentially invoked an error for moving the head off of 
the end of the tape. 


25/148 


|| Die I 


6 7 8 9 10 11 12 


Here we show a first area, say a0, including cells 7, 8 and 9. And a second 
area, say a1, including cells 10, 11, and 12. Thus, by our convention of using 
the cell to the left of an area as its address, a0 has an address of 6, while area 
al has an address of 9. Because the areas are adjacent, the address of area 
al is the same as the address of the rightmost cell in area a0. 


We know that area a0 is located to the left of area a1 because a0's address is 
less than al's. i.e. 6 < 9. 


Now suppose we delete cell 10. Though cell 10 is gone, our addresses remain 
consecutive, so what was cell 11, is now called cell 10, etc. and the diagram 
appears much as before, though area al is now only 2 in length: 


/| Die e 


6 7 8 9 10 11 12 


Now we delete all the area a1. During the deletion, and just after, the head 
will be on the rightmost cell of area a0, i.e. on cell 9. We can now say we have 
an area of zero length located at 9. 


/| GEREK 


6 7 8 9 10 11 12 


Now suppose after deleting the area a1, as just described, we continue on to 
delete area a0. Each time we delete a cell, the rightmost cell of the area at a0, 
the one that is at the same address as the address for a1, moves one closer to 
the cell used as the address for area a0. Hence after deleting the leftmost cell 
of area a0, the address for area a1 becomes 8. This is a consequence of our 
recursive definition for addresses, and would be true independent of the 
length of a1. 


/| We Titi ic 
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Area a0 is still located to the left of a1, because 6 < 8. 


Finally when all cells in area a0 have been deleted, a1 has collapsed into a0. 
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Both have the address of 6, so we have lost the order information between 
the two areas. Should we attempt to reverse the steps above, and only be 
given the machine at its final state and the addresses for the two areas by 
name, we will have to begin by guessing what the order was between the two 
areas, a0 and a1. 


We can solve this problem by adding a third level for order accounting. In 
this third level we reason in the abstract about the order between areas, even 
when those areas have zero length. 


More about the Tape 


Suppose we unmount a tape from a halted Turing Machine, and then mount 
the tape on another Turing Machine as input. Suppose we do this so that the 
second machine may calculate the length of the output created by the first 
machine. When we do this we run into some problems. 


Notice that a Turing Machine that does nothing will implement an identity 
relationship between input and output. Hence we potentially have a problem 
with preventing the input from 'bleeding through' unless we erase it. But we 
allowed that an input can be infinite, and it is not possible for a Turing 
Machine to erase an infinite area. Hence to have a clean definition while 
permitting infinite inputs, we must add an 'erase to end of tape' command to 
the machine. This is a single command that executes in one step. 
Alternatively, we can add a special symbol as an 'end of active area' marker 
to the alphabet. Should this output be used on another machine, the 'end of 
active area' marker that denoted the end of the output on the first machine, 
will then denote the end of the input for the second machine. As yet another 
solution, we can move to multiple tape Turing Machines, where one of the 
tapes is dedicated to the input. We would then have to drop and rearrange 
the tapes to make the output of one Turing Machine computation the input 
for another. This would be equivalent to having our erase command. 


In the case of an infinite input on one tape, we will need to write over the top 
of the input to make use of the tape. We can add extra states to our state 
controller to create a small buffer if need be. To avoid this we can require 
that the input be given to us with padding, or even given to us on only odd 
addressed cells, which would be the same as having another tape, and a two 
tape single ended Machine is equivalent to a single bidirectional tape. 
Neither of these solutions feels completely satisfactory, as the extended 
controller solution conflates control with data, and the second suggestion 
leads to a non-uniform approach to providing inputs. With our TM Library 
we adopt the convention of multiple tapes. 


There is no way for a second machine to know if the first machine left a big 
gap of empty-symbols in the middle of a tape. Consequently, a second 
machine that is looking for its input cannot stop scanning right at any point 
when it reads an empty-symbol, or string of empty-symbols, because 
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another alphabet symbol might exist yet further to the right. 
Here are some workarounds: 


1. We provide second level information about the length of the active 
area. This is usually how we do complexity analysis - i.e. the length 
of the input is decreed externally as a parameter, typically called 
“N”. 

2. We embed information about the end of the active area within the 
active area. This can be an address, a sequence that is recognized 
by the controller, or a special symbol. 

3. We adopt a convention of maintaining a compact tape. As such we 
have no embedded empty-symbols in the active area. Then the 
empty-symbol marks the end of the active area. 

4. We have external accounting, i.e. a type system. 


A compact area is one that has only alphabet symbols (no empty-symbols). 
We can extend this concept to say that the density of an area is the ratio of 
alphabet symbols to empty-symbols. 


The work around we use in modern computing is the 4t: one listed above, 
that of the type system. We carefully account for the length of each instance 
of data. Then we build up each larger instance from smaller ones, and while 
doing so, we add the lengths of the smaller instances to calculate the length 
of the larger instance. All programs then specify when they create or 
compose instances. The length accounting is kept in a table, which 
technically means it is on the tape. Each row of the table holds two values, 
one to locate an instance on the tape, and the other to specify its length. Often 
we include a third value to give the instance a symbolic name. We call sucha 
table a symbol table. 


Some properties of Turing Machines 


For a given Turing Machine, the input is the sole determiner of the output. 
Le. each time the same input is given, we get the same output. The input and 
output are mathematical objects; hence Turing Machines are functions. 
However, it is common in computer science to speak of Turing Machines as 
‘solving problems' rather than saying they are functions. This is because we 
often think of the inputs and outputs of Turing Machines as being something 
other than mathematical objects. For example, when a Turing Machine sorts 
sequences found on its input, we might say that it solves a sorting problem. 


Turing Machines will differ due to differing alphabets, state controller 
graphs, associated commands, start, and halt states. The choice of empty- 
symbol is inconsequential as long as it is distinct from the alphabet. As we 
will see in the later discussion on variations, Conventional Turing Machine 
variations, the choice of alphabet is not very important. For two alphabets of 
the same cardinality we can create a one to one mapping, and for those of 
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differing cardinalities we can use sequences of alphabet symbols that map to 
alphabet symbols. For example, given an alphabet of ‘T’ and ‘F’, and a second 
alphabet of ’t’, f, ‘x’, ‘z? we may make the following map: 


f FF 
t FT 
x TF 
z FF 


Then given this mapping, we may use two cells for each one cell for any {‘f’, 
‘t’, ‘x’, ‘z’} alphabet machine, and then use the only the {‘T’, F} alphabet. 


There are a countably infinite number of permutations for alphabets, state 
controller graphs, associated commands, start and halt states, hence there 
are a countably infinite number of Turing Machines that fit our definition. 
However, there are an uncountably infinite number of mathematical 
functions. Consequently, we must expect that some functions cannot be 
computed with Turing Machines. 


There might be multiple Turing Machines that perform the same function. A 
set of such machines forms a functional equivalence class. Within a 
functional class there will be a class of members related in that they all use 
the smallest number of steps when considered against the limit of input 
length. We discuss this further in the section on complexity, Complexity. 


Of special importance to computation theory is the existence among all these 
infinite Turing Machines of a class of machines that read their state 
controller definition from the tape as an input. This is the Universal Turing 
Machine class. 


Complexity 


An interesting aspect of the Turing Machine procedure is that it introduces 
the concept of stepping the machine. With the addition of some simple 
constraints it becomes possible to map the parts of the Turing Machine 
abstraction to the parts of some real machines. These constraints may take 
the form of such things as a bounds on the length of the inputs, or the 
addition of out-of-resource errors. Because such constraints do not affect the 
‘normal’ workings of the machine, the derived relationship between a Turing 
Machine step, and that of a unit of real time might not be that complicated. 
Indeed, except for some enumerable cases this relationship might even be so 
simple as to assign to a step an approximately constant amount of time. 
Because of the existence of a relationship between steps and time, 
particularly when it is a simple one, it is very interesting for us to know how 
many steps a Turing Machine will take. 


There are many ways to measure the complexity of a Turing Machine. Among 
these is something called the time complexity, which is a function that relates 
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the length of the input to the number of steps required to reach the halt state. 
To derive time complexity we typically start with a step count formula which 
maps the length of Turing Machine input to the worst case largest number of 
steps. We then consider the behavior of this formula as input length goes 
towards infinity. To get this, we take the highest order term from the step 
count formula. Conventional results are constant time, polynomial time, or 
exponential time. 


We can derive the 'worst case length of the area written or read by the 
machine during computation’ function in an analogous manner as for the 
step count function. This function is known as the space complexity. We may 
also consider the limiting behavior of this function to derive an order of 
space complexity. 


The order of time or space complexity will remain the same against certain 
variations of our Turing Machine definition. For example, if we double all the 
states by adding a second state that we always visit, where this second state 
does nothing, the number of steps would double but the functionality would 
not change. Order of complexity also would not change. A fixed time machine 
before doubling up on the states would still be a fixed time machine 
afterward. It is just that the particular fixed number of steps would be twice 
as large. A polynomial time machine would still be polynomial time, just with 
double size constants. We say that changes which do not change order of 
complexity, nor existence proofs, are inconsequential. 


Suppose we have a complete Turing Machine functionality class. We say that 
it is complete because all possible machines for implementing the function 
are in this class. Some machines in this class will have a different order of 
time complexity than others. Now we consider the set of minimum order of 
time complexity machines from this class. As the larger set was complete, the 
set built against this constraint will also be complete relative to the 
constraint. We then say that this minimum order of time complexity is a 
property of the problem being solved, rather than being a property of a 
particular machine. 


Conventional Turing Machine variations 


In the first section of this chapter we gave a rather conventional definition 
for a Turing Machine. In the prior section, Complexity, we noted that we can 
analyze Turing Machines to find their time and space complexities. In this 
section we will discuss some variations that one finds in the literature. 


A variation on the conventional Turing Machine definition is allowed when 
it can be proven that the variation never causes existence, order of time 
complexity, nor order of space complexity results to change, and in this 
respect is inconsequential. Earlier we gave the example of doubling up the 
states as being such a variation, though that is not a conventional variation. 
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Open in both directions Tapes 


Some Turing Machine descriptions describe a tape with no end in either the 
left or right directions, i.e. a tape that is that is open in both directions. 


This feature adds no richness of expression, because we can get the same 
behavior from a Turing Machine with a single ended tape. To do this we 
partition the single ended tape into odd addressed cells and even addressed 
cells. The odd cells are said to be the right side of the tape, and the even ones 
the left side. We then rewrite any tape controller based on a bidirectional 
tape to instead use the 'odd' and ‘even’ channels instead of the left and right 
sides of the tape. 


This same approach form can be used to show that multiple tapes, or even 
multi-dimensional tapes, add no expressive power. The good news is that 
such variations can be used whenever convenient, and we will get the same 
results. 


Going in the other direction, the open in both directions tape is not a 
simplification. There is still a start cell, being the cell that the head is initially 
placed on. And as noted above, the topology around this start cell is no 
different, it is just a question of the adjectives we use for describing it. 


Alphabet replacement 


Without loss of generality, we may replace the alphabet with a single symbol, 
say 's' (short for successor). This is because symbols in any alphabet can be 
placed into correspondence with a sequences of's' symbols. For example, the 
symbols of the alphabet of {w, x, y, z} can be placed into one to one 
correspondence with the sequences in the manner of {<w, s>, <x, SS>, <y, 
sss>, <z, ssss>}. We will need to put the empty-symbol between any such 
sequences on the tape, so that two sequences can be distinguished from one 
longer sequence. Alternatively we can adopt a two symbol alphabet instead 
of a single symbol, where the second alphabet symbol is an end of sequence 
marker. 


In contemporary computing we use an alphabet of two symbols, {0, 1}, and 
fixed length sequences. No end marker is needed when the sequences to be 
placed into correspondence are of fixed length. So for example, the symbols 
in the alphabet {dog, cat, mouse, fish} may have the correspondence of {<dog, 
00>, <cat, 01>, <mouse, 10>, <fish 11>}. Conventional fixed sequence lengths 
are 8, 16, 32, and 64. So for example, when the sequence length is 8, any 
alphabet of 256 symbols or less may be placed into correspondence. A 
conventional correspondence table is that of the ASCII code. The fixed length 
to be used can depend on computational context. (In contrast, UTF8 does not 
use fixed length sequences, so there must exist at least one end of sequence 
marker.) 
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Another alternative to end of sequence markers for variable length 
sequences is to externally account for sequences lengths. We call such an 
accounting system a type system. 


Leaving out the empty-symbol 


When we use a fixed sequence of {1, 0} to stand for symbols, it is expensive 
to reserve a sequence for the empty-symbol. This expense is due both to 
losing the use of a symbol in the alphabet, and in the complexity of control 
circuitry when keeping track of it. 


The empty-symbol is a property of the machine rather than just another 
alphabet member, because the tape initially has an infinite tail of empty- 
symbols. A computational Turing Machine is limited to taking a finite 
number of steps. Thus it cannot compute a tape initialized with an empty- 
symbol (or any other value). 


However we can add a constraint on all Turing Machine controllers that 
requires of controllers that they never write the empty-symbol, and always 
write an alphabet symbol to a cell before reading it. Then, because a cell is 
never read before being written, it does not matter what we write into it for 
initialization. We may even use an alphabet symbol. Consequently this 
constraint allows us to eliminate the empty-symbol. This gives us the 
following Turing Machine variation: 


1. fixed finite alphabet 

2. fixed write before read constraint 
3. variable single ended tape 

4. variable read/write head 

5. variable read value buffer 

6. fixed controlling state machine 
7. fixed left from leftmost error 

8. fixed start state 

9. fixed halt state 

10. variable current state 

11. fixed procedure for using these. 


To validate that this is an inconsequential Turing Machine variation, rather 
than a description of new abstraction that is not a Turing Machine, we must 
show two things: Firstly, that any of the now disallowed controllers never 
compute something that cannot be computed in the presence of the 
constraint. And secondly, that there are same complexity class alternatives 
for any disallowed controller. 


No need to step into the inactive area 


1. When attempting to step into the inactive area, instead keep a counter for 
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the number of steps the machine would take. Only allow reads or writes or 
head movement when the counter is no longer needed due to the head 
having moved back into the active area. 2. Just write an alphabet character 
and change the inactive area traversed into an active area. 


A finite Tape variation 


The active area on the tape can grow at most by one unit for each machine 
step. This largest growth occurs when the Turing Machine steps right and 
writes an alphabet character in every visited state. This means that for 
computational machines that start with a tape that has a finite input (active 
area), the output (active area) will be finite. This also means that space 
complexity can never be larger than time complexity. 


A fixed value is one that is provided with a Turing Machine definition, and 
does not change while the machine runs. Suppose we chose a fixed length 
Turing Machine tape. The tape would then have a rightmost cell. That cell 
would have no right neighbor, but would have a left neighbor. We would also 
add another error, that of right from rightmost. This error would be invoked 
when the controller attempted to step right from the rightmost cell. 


Consider a machine that does not step out of the active area, has constant 
space complexity, and where this space complexity is less than the fixed 
length for the finite tape - such a machine would never trip the right from 
rightmost error, and thus there would be no difference between a finite tape 
and an infinite one. 


Now suppose that we bound the length of the input, and that the maximum 
space required for such inputs or shorter ones is less than or equal to the 
length of the tape. Then again, the right from rightmost error would never be 
taken, and thus the tape would be indistinguishable from an infinite one. 
(Today we typically pad programs with lots of memory and long address 
words in hopes this will be the case.) An analogous argument can be made if 
we bound the number of steps that may be taken. 


Now consider the case where we do not fix the length of the input, nor the 
number of steps allowed, and that space complexity is such that space usage 
grows with growing input length, at least for very long inputs. For such 
machines we can always find an input of sufficient length to trip the right 
from rightmost error. 


As another approach to finite computing we can run computations twice. For 
a given input we first run the Turing Machine variation that does not step out 
of active area but still has an infinite tape. We watch this machine closely 
while it is running and find the bound on the active area. Now we can create 
a second machine that has a fixed length tape at least as long as our active 
area measurement but is otherwise the same. Now with this second machine 
we can run the same input and there will be no right of rightmost error, and 


33/148 


thus there will be no difference between having the finite tape or an infinite 
one. 


Unlike for the constant space complexity proposal, and the bounded input 
length proposal, which only work for small subsets of potential inputs, this 
‘run twice’ proposal derives a finite machine that works for any given input 
which a Turing Machine works for. Though, unfortunately, in all cases the 
second run will be moot, as we could have simply taken the output from the 
first run. 


In a variation on the run it twice approach, instead of running the first 
machine, we might instead analyze it and should we be able to surmise a 
maximum tape length, we could use that. 


In yet another approach we can extend the finite tape as needed. We create 
a control layer over the finite tape. When a step right command from the 
Turing controller invokes the right from rightmost error, the lower layer 
allocates memory, lengthens the tape, and then performs the requested step 
right. As long as this occurs in fixed time, (or of sufficient lesser order time 
than the dominate order of the time complexity), and as long as there is 
indeed more memory to allocate - this Turing Machine variation will yield 
the same order of computational complexity as one with an infinite tape. 


I propose the following Turing Machine variation: 


1. fixed finite alphabet 

write before read constraint 
variable and extendable finite tape 
variable read/write head 
variable read value buffer 

fixed controlling state machine 
fixed left from leftmost error 
fixed right from rightmost error 
9. fixed start state 

10. fixed halt state 

11. variable current state 

12. fixed procedure for using these 


CO) SIOS OT ae. C8: BS 


In our original Turing Machine model, the controlling state machine 
commands were limited to, do-nothing, step-left, step-right, write, with 
reading as an implied command. To this list we add append. The append 
command may only be called when the head is on the rightmost tape cell. 
This is not limiting because the command may be called from a state that is 
at the end of an arc triggered by the right from rightmost error. When we have 
no empty-symbol, append accepts an alphabet symbol and performs a write 
into the new cell. This is not limiting because if need be, one can always 
perform an extraneous write of an alphabet symbol. 
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With this extendable tape model all Turing Machine components remain 
finite during computation, though some may be arbitrarily large. This 
variation is more suited for creating a mapping between a Turing Machine 
and a real program running on a real machine. 


More about commands 


The Turing Machine state controller has a command symbol tied to each 
state. The Turing Machine procedure then has us take action based on this 
symbol. This is our current command set: 


1. step-right, causes the head to move to the right neighbor cell. 

2. step-left, causes the head to move to the left neighbor cell. 

3. read, returns the symbol under the head. The returned symbol is 
then used to choose the transition arc. 

4. write(x), causes the symbol x to be written to the cell that the head 
is currently on. The symbol x is any symbol from the alphabet. 

5. append(x) may only be called when on the rightmost cell. Extends 
the tape by one cell, and writes the symbol x into that cell, where x is 
any symbol from the alphabet. 


We are going to relieve the constraint that append can only occur from 
rightmost. Our new append is functionally identical to adding a cell to the 
rightmost extremity, and then shifting all the symbols over by one cell 
starting at the new cell and ending when the new rightmost has been written 
- and then doing the requested write of x on the right neighbor cell. 


We will also include the inverse function for append. delete(append(x)) 
reads x while deleting the cell that x was in. The current Turing Machine 
model can emulate this function by shifting all the symbols in cells the right 
of the head left by one, and then just not using the rightmost cell. 


We are also going to support multiplexed state controllers. Our multiplexed 
Turing Machine will have multiple heads. One for each separate thread of 
execution through the state controller. As explained in the following 
chapters, supporting multiplexing makes our machine more complex, 
especially in the presence of the delete command. However, we defer that 
discussion to the relevant chapters. 


In cases where successive states are visited in a fixed order it is convenient 
to combine the commands. We have developed the concept of a command 
statement to support this: 
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statement:: [direction] commandt [modifier] [& contract]* [arg]* 
direction:: - | € 


command::r|w|s|[a|d|mle|? 
modifier:: 0 Œ n 


command 
r read cell under the head 
w write cell under the head 


s step 
a allocate/add/append a new cell 
d deallocate/drop/delete cell 


m move, no allocation or deallocation of cells, requires fill 
e entangled copy 
# entangled copy on a new thread 


modifier 
D operate on leftmost 
Œ operate on rightmost 
n repeats n times, n provided through an argument 


contracts 
hū head is at leftmost 
hū head is at rightmost 


examples 
-s ; step left 
all ; creates a new leftmost cell 
aŭ ; append to rightmost 
sn ; step n times 


to derive a longer command, combine 
as ; append then step 
-a-s ; append to the left, step to the left 


The left direction is specified with a minus sign, otherwise the direction is 
taken as right going. So the letter s is the step-right command, and -s is the 
step-left command. The command s3 steps right three times. 


The command a appends and writes a new cell to the right of the head. We 
use two special characters from the UTF character set to signify the 
rightmost and leftmost of the tape. This one looks like a little tape with its 
left cell inked in, ID, so we use it to stand for leftmost. We use this one, Œ ,to 
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mean rightmost. Hence al creates a new rightmost cell, and all creates a 
new leftmost cell. 


In some cases it is possible to implement higher performance 
implementations for commands when the programmer tells us some 
additional information. For example al&hO@ has identical functionality as 
afl, while the programmer also guarantees that we are on the rightmost cell. 
This saves the function from having to scan to the end of the tape. 


We can concatenate the command letters into a string to summarize what 
would happen sequentially in adjacent state transitions. If these compound 
commands need arguments, then they are pulled from the argument list in 
order as they are needed. For example, as means to append, with the 
parameter for the append taken from the argument list, and then to step. 


We support multiplexing with the command e, which is short for entangled. 
This operator returns what appears to be a second independent machine, 
but this apparently independent second machine actually shares the same 
tape with the first machine. It is functionally identical to giving one machine 
two heads, and thus the ability to have state sets. 


The command esr is a compound command referring to sequentially 
applying three other commands. The e says to make an entangled copy of 
the head. The s says to step this copy, and the r says to do the read. The 
analogous esw does a write as the last step. This sort of combination of 
letters to make more complex commands was inspired by Lisp's car and cdr 
compositions.® Though this is functionally what the command does, its actual 
implementation might be completely different. 


The combination of multiplexing and cell deletion creates the hazard where 
one thread can delete a cell the head is on in another thread. We add a 
collision error continuation to our multiplexed Turing Machine interface just 
because of this situation. 


Chapter conclusion 


The modifications made to the Turing Machine to create our TTCA machine 
were inconsequential, in that for order of complexity and existence proofs 
we may swap one machine for the other and the results will be the same. 


The original Turing Machine had an infinite tape. In contrast the TTCA 
machine has a surprising property: for computational problems all of its 
components remain finite. This follows from the fact that during 
computation a machine makes a finite number of steps, so the tape can only 


sLynch, Thomas W. “Towards a Better Understanding of CAR, CDR, CADR and the 
Others” 
arXiv:1507.05956 2015-07-25 


37/148 


be expanded to be a finite size. 
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TTCA Turing Machine in Lisp 


Because our TTCA Turing Machine has finite sized components, we may 
create a TTCA Turing Machine in software, and then run it. Doing so does not 
require any hand waving or approximation. Rather we can show a one to one 
mapping of code and data in the software expression of a TTCA Turing 
Machine with components symbols of a theoretical TTCA Turing Machine. 
Consequently they are isomorphic. Such a software isomorphism is not 
possible for the regular Turing Machine due to its infinite tape. 


When given a software TTCA Turing Machine instance in a particular state, 
and its modeled theoretical counterpart in the corresponding particular 
state, after taking one step, the new states will also correspond according to 
our one to one mapping. Furthermore, because the software machine, the 
theoretical machine, and the mapping between them are all finite, it is 
possible to check with a finite procedure that our software is indeed 
following the mapping. 


If follows that we can use our TTCA Turing Machine software to make 
theoretical statements about computation. Accordingly we will learn some 
interesting things in this chapter about analysis, emptiness, data type, the 
meaning of emptiness, the properties of non-destructive vs destructive 
programming styles, and multi-threaded programming. 


Installing the library 


TM is on quicklisp, the de facto package manager for Lisp. Alternatively, 
clone the repository www.github.com/Thomas-Walker-Lynch/tm and 
then checkout the latest release tag, which as of this writing is vO.7-alpha. 
Then cd into the tm directory and run your lisp interpreter. Inside your lisp 
interpreter type the commands (load “load”) and (test-all). test-all 
should return with a message that all of the tests passed. It is possible that 
the threading tests, 'ts1-' might fail if your machine is heavily loaded or very 
slow, as they have timing built into them, but this is unlikely. Then type either 
(use-package :tm) or (in-package :tm) depending on what your 
objectives are. 


The examples in this book either come from the tm/test directories, or from 
the tm/docs/examples directory. At the time they were placed in the book, 
they executed correctly, and I have endeavored to keep the examples up to 
date. 


This is what it looks like when I follow install using the git clone method: 
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> git clone http://www.github.com/Thomas-Walker-Lynch/tm 
Cloning into 'tm’... 

remote: Counting objects: 3052, done. 

remote: Total 3052 (delta 0), reused O (delta 0), pack-reused 3052 
Receiving objects: 100% (3052/3052), 2.41 MiB | 1.52 MiB/s, 
done. 

Resolving deltas: 100% (2366/2366), done. 


> cd tm 


> git tag 
v0O.1-alpha 
v0.7-alpha 


> git checkout vO.7-alpha 
Note: checking out 'v0.7-alpha’'. 


You are in 'detached HEAD' state ... 


> sbcl 
This is SBCL 1.3.14.debian ... 
* (load "load") 


; compiling file "/home/tm/package-def/conditions. lisp" 
; compiling (IN-PACKAGE #:TM) 
... about 10 pages of these 

hooking test: TEST-TS1-5 

* (test-all) 
+ TEST-UNWRAP-O 

... about a page of these tests 
+ TEST-TS1-5 

all 78 passed 

7 

* (use-package :tm) 


7 
* (+ 10) 


T 
* 
Notice I used git tag to see the releases. At this time, vO.7-alpha is the 


latest, so I checked that out. If you want the unstable latest code rather than 
the stable latest release, leave out the git checkout command. 


The TM Library makes use of Unicode. There is no getting around it. This is 
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discussed further in the next section. In addition TM defines synonyms for 
commands such as ‘not equal’, which is the one command shown at the end 
of the transcript given above. 


Unicode usage 


For your convenience there is a file "emacs-keys" in the docs directory of the 
distribution. It sets the 


C-x g name SPC 


command to enter one of the Unicode characters that are used in the library. 
Here 'name' is a nickname.® 


So to type the character capital delta after emacs-keys has been loaded, type 
C-x g D SPC. Actually A occurs twice in Unicode, once as capital delta, and 
once as a symbol for 'increment' in mathematics. We consider the increment 
version to only be there for typography purposes. We only use capital delta, 
even when it is for an increment variable. 


I've limited the use of Unicode mostly to things that 'probably would have 
been this way had Unicode been around before’. This includes conventional 
notation and a couple of symbol extensions that were needed to facilitate the 
TM access language. 


In the file src-O/fundamental.lisp find synonym bindings for the usual 
operators and common symbols such as A, V, 2, <, A, Ø, etc. 


Specific to the library we use the character 'ID' as shorthand for ‘leftmost’. 
This is because it looks like a little tape with the inked over cell being the 
leftmost cell. In the same manner the character '@' is shorthand for 
rightmost. We use '-' in continuation function names, and 'C’ as a loop 
operator. 


The DO and Œ symbols are used in compound command names and for 
access language statements to indicate operation on rightmost or leftmost 
rather than the cell the head is on, or, to specify contracts with the 
programmer of the sort: "this function is only called when the cell is on 
rightmost." 


Since I had symbols for leftmost and rightmost I started using them generally 
to mean leftmost or rightmost wherever it was convenient. For example, to 
shorten up the names of continuations so that parameter lists would fit ona 
line. 


I would have preferred to have modified the existing C-x 8 so that such a nickname 
followed by a space would enter a character, but when delving into the emacs 
library I found it to be rather 'Unicode technical’. Perhaps someone can help with 
this. 
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(global-set-key (kbd "C-x g d SPC") [?6]) 
(global-set-key (kbd "C-x g D SPC") [?A]) 
(global-set-key (kbd "C-x g SPC") [?y]) 
(global-set-key (kbd "C-x g SPC") [?T]) 
(global-set-key (kbd "C-x g | SPC") [?A]) 
(global-set-key (kbd "C-x g L SPC") [?A]) 
(global-set-key (kbd "C-x g p SPC") [?n]) 
(global-set-key (kbd "C-x g P SPC") [?N]) 


(global-set-key (kbd "C-x g > = SPC") [?2]) 
(global-set-key (kbd "C-x g < = SPC") [?S]) 
(global-set-key (kbd "C-x g ! = SPC") [?+]) 
(global-set-key (kbd "C-x g neq SPC") [?#]) 
(global-set-key (kbd "C-x g nil SPC") [?0]) 


(global-set-key (kbd "C-x g not SPC") [?-]) 
(global-set-key (kbd "C-x g and SPC") [?A]) 
(global-set-key (kbd "C-x g or SPC") [?v]) 


(global-set-key (kbd "C-x g ex SPC") [?3]) 
(global-set-key (kbd "C-x g exists SPC") [?3]) 
(global-set-key (kbd "C-x g all SPC") [?v]) 


(global-set-key (kbd "C-x g do SPC") [?C]) 


(global-set-key (kbd "C-x g rm SPC") [?0]) 
(global-set-key (kbd "C-x g Im SPC") [?I0]) 


(global-set-key (kbd "C-x g cont SPC") [?>]) 


(setq lisp-indent-offset 2) 

(modify-syntax-entry ?[ "(]" lisp-mode-syntax-table) 
(modify-syntax-entry ?] ")[" lisp-mode-syntax-table) 
(modify-syntax-entry ?{ "(}" lisp-mode-syntax-table) 
(modify-syntax-entry ?} "){" lisp-mode-syntax-table) 
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Synonyms 


(defmacro defsynonym (old-name new-name) 

"Define OLD-NAME to be equivalent to NEW-NAME." 

` (defmacro ,;New-name (&rest args) *(,',old- 
name ,@args)) 


(defconstant @ nil) 


(defsynonym /= #) 
(defsynonym <= <) 
(defsynonym >= >) 


(defsynonym not ~n) 
(defsynonym and A) 
(defsynonym or v) 


(defsynonym string/= string+) 
(defsynonym string<= strings) 
(defsynonym string>= string=) 


(defsynonym lambda A) 


Some reader macros 


Because of our method of continuations, and our adoption of workers 
(discussed in chapter Streaming) we have ended up with some extended 
parameter lists. Also because of our allocate and deallocate options on the 
tape machine, we end up with some clean, but destructive operations. In the 
TM Library I reduce this burden by providing some reader macros to make 
list creation a little bit easier 'q', and 'L' or {...}. And to make function 
evaluation a little simpler, [...]. 


q - anonliteral quote 


In Lisp a quoted list is taken as being literal. However the result of modifying 
a literal is undefined, and often leads to bad results. Hence we provide the 
macro q which returns a quoted list which is not a literal. 


* (qab c) 
(A BC) 
{...} - unevaluated list 


When a form enclosed in parentheses, ( ... ), is evaluated the head is taken as 
the name of a function, looked up and called. The list members are also 
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evaluated, and then passed as arguments to said function. If we don't want 
the head treated specially, but rather just want to define a list, we can trick 
the evaluator into giving us a list by padding the list with a front item of 
#'list, which is the function to create a list. 


* (list 1 2 (+ 1 2)) 
(1 23) 


We have defined a macro called L that like list, creates a list, but which 
also has some extra functionality. 


*(L12(+12)) 
(1 2 3) 


We also provide a reader macro over braces. 


* {12 (+12)} 
(1 23) 


If the apparent function open, #'o, appears inside of a call to L, then the 
arguments of the #'o function are included directly in the resulting list: 


* (defvar a {1 2 3}) 
A 


* (defvar b {4 5 6}) 
B 


* {a (o b)} 
((123)456) 


Quoted non-literals can also occur within such an L list: 


* {a (0 b) (q a b)} 
((123)456 (AB)) 


* {a (o b) (o (q a b))} 
((123)456AB) 


L is a little bit like quasiquote turned inside out. Whereas the default in 
quasiquote is to quote items, and a comma operator turns that off, the default 
in Lis to evaluate items, and a q operator turns that off. Quasiquote has an 
@ marker to open up lists, while L has an 0 operator to open up lists. Note 
however, inside of a quasiquote we could get in trouble if the name of a 
variable starts with an @ character. If such a variable appears after a comma, 
quasi quote will consider the variable name without the @ sign is to be 
opened and included. This is an example of where 'in-band signaling’ causes 
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us to have to make rules to leave values out of our data set (or in this case a 
character out of variable names). There is no analogous problem with the o 
operator because it only appears in the function channel. 


[...] - head is variable holding a function name 


In Lisp the head of an evaluated list is taken as a function name. Consider this 
example that curries a two parameter function into a unary function by 
replacing giving one parameter a constant argument of 3:7 


(defun curry-three (f n) (f n 3)) ; has errors 
When we compile this function we get two errors: 

The variable F is defined but never used. 
and 

undefined function: F 


The first f is in the parameter list of the function definition, so it is taken as a 
variable name. In contrast the f in the body is at the head of an evaluated list, 
so it is taken as a function name. Hence there is a disconnect, and we get 
error messages describing this disconnect. 


The Lisp operator #' indicates that the symbol that follows is a function 
name to be taken literally, and not a variable name. This gets our function 
name into the data space for use as an argument. The Lisp function funcall 
accepts as a first argument the name of a function to be called, while the 
remaining arguments are passed through to said function as its arguments: 


(defun curry-three (f n) (funcall f n 3)) ; good syntax 
(defun plus (x y) (+ x y)) 

(curry-three #'plus 2) 

5 


Now in the definition for curry-three we do not get interpreter/compiler 
errors. This is because f is consistently used as a variable name. funcall will 
use the value of the variable f as the name of a function to call. 


In the second line we define a function to pass into curry-three. I just put 
something simple here for sake of discussion. plus is defined to be a function 
that takes two arguments and sums them. 


In the third line we use the #' operator to tell Lisp that plus is a function 
name to be used literally as a value.® This will be a value passed into curry- 


‘curry example is expanded from that given in “The Common Lisp Cookbook” see 
http://cl-cookbook.sourceforge.net/functions.html 

8An implementation might use a different type of reference to the function than the 
functions name, such as a pointer to an entry in a symbol table, or perhaps a pointer 
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three no differently than had we put a number or string instance as an 
argument. Inside of curry-three the #'plus becomes the value of the 
variable f. Then the funcall function will use this value as a function name, 
and then call it. 


This is how function pointers are handled in Lisp, a language that does not 
have explicit pointers. 


Actually we didn't need to define the function plus, because '+' is already a 
function. We don't have reserved operator symbols in Lisp, instead we just 
have loose rules on what can be used for function names. 


(curry-three #'+ 2) 
5 


We introduce a shortcut with the TM Library. Normally a list to be evaluated 
is in parenthesis, and its head is taken as a function name. With the TM 
Library loaded, when a list in square brackets is evaluated, the head is taken 
as a variable name, and the value of this variables is the function to be 
loaded.? 


* (defun curry-three (f n) [f n 3]) 
CURRY-THREE 

* (curry-three #'plus 2) 

5 


We implemented this feature with a reader macro which simply turns the 
square bracket list into a regular list and inserts the funcall as its head. This 
occurs before the Lisp evaluate phase sees the syntax. 


Here is another example. Suppose that instead of passing #'plus in as an 
argument, that we first assigned it to a variable and then pass the variable 
value as an argument: 


* (defvar our-fun #'plus) 
* (curry-three our-fun 2) 
5 


Here the value assigned to the variable our-fun is a function name. The 
variable is then used as any other, and its value is passed in as an argument 
to curry-three. Note, all arguments are evaluated before the function is 
called, so variables are replaced with their values. And as we know, inside 
curry-three, funcall will take the function name value from the 


to the function. All that is required is that #' produces a value that can be used by 
funcall. 

°One might think of this as indirect addressing of the function. Square brackets have 
traditionally been used for such addressing. 
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corresponding parameter and call it as a function. 


Things can become a little confusing when the variable name has the same 
name as the function. 


* (defvar plus #'plus) 
* (curry-three plus 2) 
5 


Here #'plus is the function name as data. When plus is the head of an 
evaluated list is a function name, otherwise it is a variable name. It is little 
wonder that the Lisp dialect Scheme put function names and variables name 
in the same space. However by doing so they had to provide some automatic 
conversions between variable names and function names. 


Summary of list reader macros 
When evaluated: 


(q f g h) - non-literal list of symbol literals, f, g, and h. Unlike for quote, it 
is safe to modify the resulting list. 


(o f g h) - not a list, rather f, g, and h are to be included in a {} form that 
contains this form. E.g. { (0 1 2 3) } isthesameas {1 2 3}. 


{f g h} - list. f, g, and h are variables that are replaced by their values. 


(f g h) - function call. f is literally the function name. g and h are variables 
that are replaced by their values, and then passed as arguments. The result 
of the function replaces the entire form. 


[f g h] - indirect function call. f is a variable replaced by its value, and that 
value is then looked up and called as a function. g and h are variables 
replaced by their values and passed as arguments. The result of the function 
replaces the entire form. 


Continuations 


Concept 


Conventionally programmers think of programs as linear sequences of calls 
to subroutines, where each subroutine has a fixed entry and exit point. Even 
decisions made in ‘if blocks’, are considered to only create hiccups by 
skipping smalls of block of code. In the TM Library we move to a non-linear 
model by passing into some functions multiple exit continuation options. 


Here is a simple mock-up of a read function. This is provided only for 
purposes of our discussion on continuation functions. This routine either 
returns a result or a status code. Functions that do this sort of thing are 
common. Sometimes the status code is a null pointer, sometimes it is in a 
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separate variable. 


(defun mock-read (n) 
(if (= n 0) ; zero marks end of stream 
"eos 
(if (oddp n) ; say the Isb is a parity bit 
‘parity 
(write-to-string n) ; convert n to a string and return it 


))) 


In the calling code we first check the return for a status symbol, and if one is 
not found, and thus all is OK, we use the result. The check of the result for a 
status code is an inverse decision tree to that found in the function where the 
status was set. In this inverse tree we unwind what happened in the function 
so that we may properly handle the condition returned by the function: 


(defun use-mock-read (n) 
(let( 
(result (mock-read n)) 
) 

(case result ;unwind the status code 
(eos (print "end of stream")) 
(parity (print "parity error")) 
(otherwise (print result)) 

) 

t 


)) 


We might have instead thrown conditions and used a handler. We would still 
have a redundant decision in the form of checking that we catch the correct 
condition, and the overhead would be even higher. 


In contrast, when we have multiple exits the redundant tests are not 
necessary. Here is a new version of mock-read where a list of continuation 
functions is passed in as arguments. Here I have given the list a one character 
name, namely “>”. I pulled this name from the Unicode character set, and 
from hence forth refer to it as the ‘continuation’ character. This is not an 
operator, rather it is just a character as any other. In a prior version we 
spelled out a 'cont-' prefix on every continuation variable name, but that 
approach took up so much space in parameter lists that line wraps 
sometimes muddled the source code. We have come to appreciate the more 
compact form. 
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(defun mock-read-with-continuations (n &optional >) 
(destructuring-bind 
(&key 
(ok #'echo) ; if all goes 'ok' 
(eos (be 'eos)) ; we don't like zeros! 
(parity (be 'parity)) 
&allow-other-keys 


) 


=> 
(if (= n 0) ; zero byte marks end of stream 
[>eos] 
(if (oddp n) ; parity check 
[>parity] 
[ok (write-to-string n)] 


)))) 


After the list is passed in, we use destructing-bind to parse the list. 
destructuring-bind is the same function that is used by eval to bind 
arguments to parameters. Consequently we use the same syntax here as for 
specifying function parameters, and &key gives us keyword arguments, etc. 
Keywords are useful for function arguments because lambda function 
arguments can get quite long. 


Here the continuation listis used to pass three arguments, which are bound 
to the parameters ok, >eos, and > parity. It is legal to not specify all the 
keywords when passing in arguments. The #'echo, (be 'eos) and (be 
‘parity) are defaults to be used for missing arguments. Generally we chose 
the default arguments to give us the old fashioned type of behavior - i.e. the 
behavior we would get if continuations were not used. 


By adopting the convention of prefixing a '—>' onto continuation function 
names we can easily recognize that they are function continuations when we 
see them inside the function body. Note the lonely '-' is the name of the 
continuation list as a whole. It isn't a separator. 


Because we are building this over Lisp, the function exits are provided as 
arguments, and explicitly called. As a convention we always provide the 
continuations on the tail of the argument list. We often tip our hat to the old 
concept that one exit is somehow normal, by calling it ok. 


The very last thing that a function that is passed continuations must do is to 
call at least one of the continuation functions. The continuation result cannot 
be used in the function, and indeed no processing can occur after the 
continuation is called. This is currently by contract with the programmer but 
will likely be built into a macro in a later version. There can be no exceptions. 
Each thread either terminates in the function or leaves the function through 
a continuation. 


Here is a call to mock-read while using the multiple exits subroutine: 
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(defun use-mock-read-with-continuations (n) 
(mock-read-with-continuations n 


{ 
:90k (AÀ(result) (print result)) 
:=>eos = (A() (print "end of stream") t) 
:> parity (AQ (print "parity error") t) 
a) 


Note that the second argument passed into mock-read-with- 
continuations is the list of continuation functions. Each function is 
preceded by its keyword. We could have provided these in any order. the first 
continuation function accepts the result and prints it. The second one 
informs us of the end of stream, while the third says there is a parity error. 
Each of these represents a branch the code might take after exiting the read. 


There are some useful stub and error reporting functions defined in the 
distribution file functions.lisp. You might notice that we referred to #'echo 
as a default parameter above. 


(defun do-nothing (&rest x) (declare (ignore x))(values)) 
(defun echo (&rest x) (apply #'values x)) 


(defun be (&rest it) 
(A (&rest args)(declare (ignore args))(apply #'values it)) 
) 


(defun cant-happen () (error 'impossible-to-get-here)) 


(defun alloc-fail () (error ‘alloc-fail)) 


do-nothing lives up to its name. It accepts any number of arguments, 
performs no operation and does not return anything. echo returns its 
arguments. Actually it is just a synonym for values. be returns a constant 
independent of what is passed in for arguments. cant-happen throws an 
error saying it is impossible to get to this function. This is useful for finding 
mistaken assumptions. alloc-fail throws an error saying that allocation 
failed.10 


All the functions in the library use continuations for end cases rather than 
throwing errors, because what is one person's error, is another person's 
opportunity to run some more code. In particular, this is how we add code 
for handling such things as continuing from left from leftmost, and right from 
rightmost. Continuations are important for achieving Turing Complete 
computing. 


When we pass tape machines into functions, we may have stateful changes 
to the input arguments. For example, the movement of a tape head. We like 


‘OHowever, in the version of the library as of this writing you will see the Lisp 
allocation condition rather than calls to the allocation fail continuations. 
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to keep such state changes transactional relative to our end case 
continuations. Consider, as an example, if our read function was reading from 
a tape machine that was passed in. If we see the end of stream marker, we 
will leave the head on the passed in machine resting on the last character of 
the tape rather than on the eos marker. The fact we took the end of stream 
continuation is information enough that we reached the end of stream. There 
is no reason to lock in a particular implementation for marking the end of 
stream. 


Regular &optional and &rest arguments 


Our use of &optional just before the continuations list parameter constrains 
the programmer from using &rest for regular parameters. Should the 
programmer need to provide a variable number of regular arguments, then 
he or she should pack them into a list. The '{..}' macro makes this simple to 
do. 


(defun f (a b &rest c &optional >) ; bad parameter syntax 


(defun f ( a b more-args &optional >) ; good parameter syntax 


Then within the function the list can be unpacked using destructuring-bind 
where &rest and &optional, etc. are available. Indeed this is the approach 
we have used for the list of continuation functions, and we have already 
provided an example. A call would then look like this: 


(f a-val b-val {cO c1 c2} {cont-ok cont-badO cont-bad1i}) 


Classes 


The great fashion craze in computer science after that of structured 
programming was arguably that of object oriented programming. This 
chapter discusses the terminology used for object oriented programming 
with Turing Complete computing and the TM Library. Lisp's object oriented 
programming extension is called CLOS.1! We explain in this chapter how the 
CLOS approach is the dual of that used in a number of other languages such 
as C++. 


Format, instance, type, and Abstract Data Type 


As we noted in the section, Alphabet replacement, data type is related to 
formatting. When we use format consistently, writing one variable's value 
will not corrupt another variable's value. Hence, any system for consistently 
applying data formatting is a kind of memory protection system. 


The fundamental parameter for data format is the length of an area of 
memory required for holding formatted data. We may apply operators to 


Common Lisp Object System 
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compose fundamental format descriptions to create compound format 
descriptions. Such operators are provided in languages that facilitate user 
defined type. As examples, in the C language, we have struct which 
compounds format of differing length, and [:] which compounds formats of 
the same length. 


In high level languages we give names to formats. This facilitates the 
consistent use of format in assignment, across subroutine boundaries, and 
across other module concepts. By using names for formats, a compiler tool 
or a human programmer, can ask the 'is-a’ question about an area of memory 
and the answer will be a symbol. Though, as it turns out, the compiler tool 
and the human may have different goals when asking 'is-a' questions, or 
when stating 'is-a' facts. 


We say that data in an area in memory that is accessed according to a given 
format is an instance of that format. Instantiating a compound format creates 
a composite instance made of the smaller instances that correspond to the 
components of the compound format. 


Now here is an interesting development. Suppose we describe two formats 
that are identical. Perhaps we determined they were identical by having 
analyzed the lengths of the components and any operators used when 
compounding them. However, we also give the two formats descriptions 
different names. In computer programming this happens often. Say one is a 
complex number format, and the other is a vector of two components format. 
As we think about these differently we defined them separately. As a result 
of having defined these with different names our compiler may refuse to 
copy data even though there is no danger of corrupting memory. In this 
situation the format name is carrying with it a distinction that is independent 
of that of the format description, and this information is not otherwise 
available to the compiler. 


As it turns out, 'is-a' is a richer operator for humans than it is for compilers. 
Despite it being unnecessary for protecting memory we often keep such 
different names. When we do this, we are doing something more with these 
names than just describing format; hence we create a new term that 
subsumes that of format. We say that we have created a data type, rather than 
a data format. 


We might notice that there are actually three entities in our example, there 
is a complex number type, a two component vector type, and a format for 
holding a number pair. 
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Figure 2: Type, Format, Instance 


A more descriptive term for data type would be something like format with 
component name translation. In Figure 2: Type, Format, Instance we show 
user defined, or programming language defined, component names used 
within types being mapped to a format. Here we diagram our complex vs. 
vector example and show that they share the same format.12 


We must have access functions to be able to use the formatted data. We will 
only need read and write.13 In this example fmt-read will accept an instance 
base, an offset value from the format table, and then return instance from 
memory. fmt-write will accept an instance as a third argument and copy that 
instance to memory. 


We have included fmt-read and fmt-write as part of the type definition. This 
was necessary because type subsumes the definition of format, and format 
is about consistent use, and thus protection, of memory. Consequently, the 
definition of data type is part logic, and part implementation. We would like 
to change this, because from the perspective of logic functionality, the calling 
program has no vested interest in the implementation. Suppose we pull out 
the format and access functions from the definition of type. We might then 
have two different programs that had the exact same logic, but would 
compile into different implementations. When we separate out the 
implementation, we create an abstract notion of data type."4 


For example, suppose we have an instance packed in memory for our 


In most compiler implementations there would be a table for each type, and format 
would not be explicitly delineated. 

For a multithreaded environment we will also need some sort of locking or atomic 
read modify write. 

‘4K nuth uses the term abstract data type to describe type defined through functions 
rather than format. 
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complex number type, such as that instantiated from format declared with 
struct in C. Our access read function accepts a given field name, say either re 
or im, which are symbols, and then returns the contents of that field. Write 
does the inverse action. Now compare this to using C++ STL map, where the 
access functions accept field names as keys, and then traverses a red black 
tree to access the instance. In both cases the call to the read function accepts 
the same symbol and reads or writes the same instance. 


A C struct and an STL map are decidedly different formats. Hence, they 
cannot be the same data type. However, either can be used as the 
implementation for returning values given keys (or field names), i.e. they can 
be used for implementing the same abstract data type. 
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Figure 3: Abstract Data Type 


In Figure 3: Abstract Data Type the programmer sees only a symbolic form 
of the type and a read/write interface. Different implementations may be 
created by providing different compatible implementation types. Though the 
programmer creating the logic might not see it, we still need a data type for 
the implementation. The appropriate field in the data type will be lined up 
with the key passed in from the abstract type, and then access will proceed 
as for data types. 


With CLOS, a new type is defined by giving a type name and field names in 
the def-type form.15 CLOS also provides an implementation, but we emulate 


!5def-type is an alias we provide for CLOS's defclass. I would like to avoid conflating 
the concepts of type and class, so have provided this synonym. 


54/148 


abstract data type by trying not to think too much about it, and instead 
concentrating on the program logic.1¢ This is Lisp so there will be many 
parentheses. Here is a type called complex-number: 


(def-type complex-number () ((real-part) (imaginary-part))) 


You will see more examples of this in the remainder of the book. Note that in 
CLOS the ‘field names’ are called 'slots'. I guess they didn't want to conflate 
what looks like an abstract type declaration using def-type with the 
combined type and format building of C's struct or perhaps with fields from 
database records. 


Inheritance 


The second parameter of the def-type macro form provides a place to put a 
list of other types, and then the slots from those other types will be 
automatically included in the new type. This process of including slots from 
other types is called inheritance. We say the new type is more specialized 
than the types it inherited slots from. If we do not want to inherit slots from 
other types, as was the case in the complex-number example, give an empty 
list as the value to bind to the second parameter, i.e. give an empty list as an 
argument. 


Typically instances of a specialized type can be used by functions that were 
designed for instances of the more general type. This is because all the 
required slots will be present in the more specialized type, while code 
written for the more generalized type ignores slots it does not use. In fact, 
CLOS will assume we want to do this, and during dispatch it will fall back to 
type matching against more general types when it does not find a match on 
the specialized type. 


This explanation of extra slots being ignored works well when arguments are 
passed by reference. However, when arguments are passed by value, we can't 
ignore each instance's actual size. In Lisp all instances created from user 
defined types are passed by reference. 


There can be multiple specializations for a given type, each adding different 
additional slots. This happens in the TM Library. 


Abstract Functions and Function Classes 


When programming in a language like C we are forced to write our functions 
with format correct data usage embedded in them. Functions are then linked 
by name. However, sometimes we would prefer a more abstract, or more 


16Alas, perhaps someday the compiler will be smart enough that such an approach does 
have such a performance impact. Though sometimes flexibility does translate into 
performance. It certainly translates into faster development time. 
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generic behavior. Say for example, we want a print function that takes an 
argument, we don't care of which type, and then outputs a Unicode image of 
the argument. 


In such cases the functionality is defined in the abstract, but how can we 
implement this in a program? One solution is to implement many print 
functions, and then have the tools chose which print function to link based 
not just on the function's name, but also on the type of its argument. 
Accordingly we place a simple call of the form print(x) in our source code. In 
a compiled and linked language, the linker will select the print function to be 
used based on the type of x. Thus print(5) will call a function that converts 
integers to strings and copies them to the output, while print(“Tom”) will call 
a different print function that copies a string to the output. 


The set of functions used are equivalent in the sense that they all provide the 
same abstract functionality. When we put functions that are equivalent 
according to some property together, we have something called an 
equivalence class. In this case it is an abstract function equivalence class. We 
will shorten this term to function class. In our print example, print is the 
name for the class rather than for a particular member of the class. 


In our version of CLOS a function class is declared using a def-function-class 
form.!” If we were trying to create the behavior as described for our print 
example, then we would define the print class as: 


(def-function-class print(x)) 


In C++ we can implement function classes, but for some unknown reason the 
language does not give us a facility for declaring them. 


Type Classes 


Alternatively we can create the dual of this concept, and place all of the 
functions that accept the same type of argument into a set. This set would 
define a type of argument equivalence class, or type class for short. We name 
a type class after the type that is used to define it. For example, if we defined 
functions that operate on a type called, 'complex-number’, and put them all 
in a set, we would have a complex-number type class. 


The set of functions that operate on formatted data gives that data its 
personality. This is to say, that one can analyze the functional use of data and 
recover a definition of abstract data type. Though the definition of abstract 
data type recovered from analysis will be missing any unused fields. For 
example, suppose we never use the second component of a vector pair. 
Analysis would then not see it. This difference might affect our future choices 
when modifying a program. As there might be differences, when we want to 


'7def-function-class is an alias we provide for CLOS's defgeneric 
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make a distinction, we will use the term analytical type. Recovering type from 
analysis is important for languages where we do not declare type explicitly. 
A type class will contain the functions that define an analytical type. 


It is interesting to notice that though we abstracted data type, it wasn't 
enough, and we had to abstract functions as well. This makes the expression 
of functions richer than that of the expression of data. Theoretically we know 
that there is a second order infinity number of mathematical functions, but 
only a first order infinity of data that can be expressed as computer variables. 
So this experience we have had in abstracting data and functions parallels 
the theory. Yet computer functions are coded as data. Perhaps this explains 
why functions tend to be longer and less regular (more complex) than the 
data type declarations for the data they work on. 


CLOS does not require us to define type classes, but other languages, such as 
C++, place a great deal of emphasis on it. In C++ a programmer includes the 
literal class, followed by an open brace, a list of all the member functions, 
and then a closing brace. This works well the defining type applies to a single 
argument, but, when there are two or more typed arguments the functions 
will belong to multiple type classes - so where should the programmer 
declare them? Surely we don't want the programmer to have to declare the 
same function more than once! In such cases C++ drops into a ‘friend 
function’ approach. We would need this, for example, to define binary 
operators on our complex-numbers. When calling a friend function there is 
no prefix argument preceding a dot, instead all arguments appear in the 
argument list. 


It is probably because of this multiple argument issue that CLOS does not 
have a construct for declaring type classes. 


Dispatch 


Though CLOS does not facilitate the explicit definition of type classes it does 
select each function based on types of its arguments, in a manner similar to 
selecting friend functions in C++, though because it is Lisp, this is done 
dynamically. The process of matching of the function name, matching of 
argument types with possible generalization, and calling a function is called 
dispatching a function. Function dispatch incurs considerable run time cost. 


Though CLOS does not have built in support to help the programmer declare 
the type classes, it will throw warnings if the programmer does not declare 
the function classes. 


Lisp does not normally perform argument type matching, so we need a new 
form for declaring functions with typed arguments.18 One defines functions 


'8the term 'form' in Lisp is analogous to 'statement' in other languages 
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with type signatures by using the defun-typed form.!9 
Here are a couple of examples: 


(defun-typed print ((x integer) ...) 
(defun-typed print ((x string)) ...) 


Note that the ellipses indicate that code was left out. They are not part of the 
functions. The parameter list consists of pairs. The first in the pair is the 
parameter name, and the second is its type. Hence the first print function 
accepts a first argument of type integer. If an argument to be bound to the 
function parameter may be of any type, it is listed in the old fashioned 
manner without a pair. 


The Tape Machine Data Type 


Introduction 


The TM Library provides functions for iteration and continuation over 
containers. The tape machine interface is described in this section. This 
interface is abstract, constructed from a combination generic functions and 
declarations without definitions. 


Functions in the tape-machine class may be used to traverse through a 
container, access instances held in it, and to handle end cases. Loop 
constructs follow naturally as quantification operations on containers. 
Unlike for the standard loop structure in Lisp the unprocessed parts of lists 
are accessible after exiting a loop, and may be used to continue the operation 
after a fix. Unlike for stream libraries, the tape machine has random access 
capabilities between continuation limits. 


The iteration model is constructed from formal concepts, most that have 
been well studied in computation theory. Tape machine based computation 
is a hybrid between that Turing Machine and a Lambda Calculus, with the 
Turing Machine used for iteration within a region, and then continuation 
functions, lambdas, invoked for extending the boundaries of the region. | 
make the claim that there are no library architectural limitations for working 
with arbitrary long containers or computations. 


If for no other reason, the formal derivation is useful because it guides the 
terminology. As it turns out, there is an explosion in the number of end cases 
when dealing with containers and generalized iteration. This formalism 
provides a language for talking about them. 


The TM Library may be used as a tool when writing Turing Complete 
programs. The current Common Lisp implementation of the library will be 
subject to the limitations of the Common Lisp interpreter or compiler. One 


'Sdefun-typed is an alias we provide for CLOS's defmethod 


58/148 


will notice limitations for such things as address width and available virtual 
memory. The general Turing Complete architecture lifts the limitation on 
address width, and facilitates a graceful approach to the limits of the 
underlying physical memory. Even after depleting physical memory, one 
might continue again later, as the Turing Complete architecture puts no 
limitations on a physical implementation that supports the hot-swap 
addition of memory. 


Nomenclature 


Many implementations of the tape-machine type have a tape, and indicator 
head (or just head) fields - where these are placed into analogy with their 
namesakes in the Turing Machine model. 


Conceptually, or actually, a tape consists of an array of one or more cell(s). 
Each cell holds exactly one instance, where such an instance is either a) a 
subspace or b) data for which the tape machine user knows or can recover 
the type for. 


For a projective tape machine that has two or more cells, all cells, except two, 
have a left neighbor and a right neighbor. The two potentially special cells are 
called leftmost, which has only a right neighbor, and rightmost, which has 
only a left neighbor. When such a machine has only one cell, well, that cell 
has no neighbors. 


For the TM Library we allow the inclusion of other topologies. For example, 
in an affine machine of three cells or more, all cells have both distinct left and 
right neighbors. This is accomplished by hooking the cells in a ring. When 
there are only two cells in such a machine, then for each cell, the other cell is 
both its left and right neighbor. For a one cell affine tape, the one cell is its 
own neighbors. 


A tape machine might not be implemented over a container. Instead, for 
example, it can be an abstraction that is maintained through function calls, 
in which case even a projective tape might appear to be open and not have a 
leftmost or rightmost. 


Instances are the things we put into cells of the tape. Instances are not part 
of the container topology, i.e. they do not have connectedness. Hence they do 
not have neighbors or location. However, they may be placed into cells, and 
cells are connected. So, for example, rather than saying something like “the 
leftmost instance”, it is better to say, “the instance in the leftmost cell.” 


The terminology of left and right stems from the convention in mathematics 
for number lines. Here the cell addresses fall on a number line. Going right 
on the number line leads to higher addresses, and going left to lower 
addresses. 


Putting the head on a specific cell is to cue the head. A number of cue 
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functions are defined in the library: park, cue-leftmost, cue-rightmost, cue-to, 
and sn. cue-to takes an address, whereas sn steps the machine (takes a first 
difference in addresses). The distance argument is passed 'in a box’ i.e. by 
reference so that it can be modified to show the steps remaining should we 
attempt to step beyond the rightmost or leftmost cell. 


Let us explore some nuances of the nomenclature that we have defined. 
Heads are either stepped, cued, or parked. We do not apply other motion 
verbs to heads. In contrast, instances are said to be moved or copied. 
“Stepping a machine” means that we follow through one iteration of the 
machine execution procedure, so it is possible that, depending on the 
command issued, that when we step the machine, but not step the head. 
Though one thing is for sure, when we step the machine, the head can step 
at most one cell. Only instances can be moved, so saying that "a tape machine 
moved", would mean that said tape machine was an instance held as data, 
and that data was moved to another cell. 


Primitive functions are declared in the decl-only source files, and functions 
with generic implementations. It is sufficient when defining a new tape 
machine specialization to provide an implementation for each of the function 
class declarations found in the decl-only files. Functions defined in other files 
are built up from these primitives. Functions may be defined with 
parameters. At run time arguments may be provided with the function call. 
When the &optional and &rest keywords appear in the parameter list it is 
possible that a function is called with fewer arguments than the number of 
defined parameters. In such a case the specified default arguments are used. 


Tape machines may have status. Status is typically whether the machine is 
abandoned, empty, parked, or active. In contrast, Tape Machine state consists 
of the variable parts of the machine including the head location and the 
instances on the tape. Though status is a kind of state, it is a state about an 
underlying machine, which is a higher level concept. 


Modifications to head location, or to the tape contents are said to be stateful 
changes. Modifications that add or remove cells from the tape are said to be 
structural. (Adding cells to the tape is called allocation. Removing cells from 
the tape is called deallocation.) We know from experience that programs that 
only modify state tend to be more straightforward and have fewer bugs than 
those that modify structure. Yet there always seem to be cases, such as 
gathering results, where structural changes are more convenient. Even 
Scheme allows one to use a cons cell to lengthen a list, albeit only from one 
end. 


When we need to specify boundaries, we try to use inclusive bounds so that 
we will not need to represent addresses that are outside of our address 
space. Our use of quantifiers in place of loop structures facilitates this 
approach. 
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Variables that hold tape machines often have a suffix of '-tm'. We think of this 
as being analogous to putting an 's' on the end of a world to make it plural. 
Take for example, suppose a tape machine holds instances that describe 
trucks. Perhaps the truck descriptor holds an id for a truck and miles since 
its last maintenance. The specifics of the descriptor are immaterial here. We 
might call said tape machine 'truck-tm’. This implies that that one or more 
truck descriptor instances, or for a status machine, zero or more truck 
descriptor instances are referenced by said variable. We do not say 'trucks- 
tm’ because that is redundant. Marking the variable with a '-tm' already 
implies plurality. Likewise we do not say 'trucks’, firstly because a simple 's' 
is easy to glance over, and secondly because it doesn't imply what interface 
can be used for accessing the referenced container. 


The typical coding form is to do some work, and then to step. If the machine 
steps right of rightmost, or left of leftmost, then we do something else that is 
appropriate for this end case. 


Stepping (iteration) 


Each tape machine tape is typically bounded. Exceptions exist for abstract 
machines implemented through functions. Such machines may have tapes 
that appear to be infinite or looped. Also, affine machines have circular tapes, 
and thus no bounds. 


To step a tape machine, say one called tm, one calls the function s and passes 
it the tm instance and optionally two continuations. Upon striking a bound 
a continuation function is executed, and that continuation function may 
overcome the bound, or perhaps take some other action. Here the two 
continuations are called cont-ok and cont-rightmost: 


(s tm {:>0k cont-ok :->rightmost cont-rightmost}) 


The function name, s, stands for 'step'. The tape machine, tm, is a 
combination of iterator and container and s causes the internal iterator to 
step. If we put the container into correspondence with a Turing Machine 
tape, the internal iterator with the tape head, and put the program, which is 
external to our container, in correspondence with the Turing Machine 
controller; then s can be thought of as stepping the head ofa Turing Machine. 


ok stands for 'continue OK’, while >rightmost stands for 'continue after 
stepping right of rightmost’. At any point in time our container will typically 
have a bounded size. When this bound is struck, we call rightmost so that 
the program has an opportunity to overcome this bound and implement 
Turing Complete behavior. 
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(s tm 
{ 
: 0k (A()(print "everything went well")) 
:> rightmost (A()(print "go get more data!")) 


}) 


For convenience we provide some general purpose continuation functions. 
One of these is the function be, which returns a function that returns a given 
value. Hence, 


(s tm {:-ok (be t) :>ok (be @)}) 


will return true, t, should the step succeed, and return false, Ø, should it not. 
These are in fact already the defaults. When used in this mode, s must be 
tested after the return, so we end up doing tests twice: once inside the step 
function, and once outside. 


It is a common mistake for beginning users of the library to put a form that 
evaluates to a value as an argument, rather than a form that evaluates to a 
function. The following, for example, gets errors: 


(s tm {:>0k t :>rightmost ø}) ; bad semantics !! 


Tape machines can also be implemented over functions rather than 
containers. So for example, we can have a tape machine that represents the 
Natural Numbers. Each time it is stepped it just goes to the next Natural 
Number. There will be no rightmost cell, so a step call might appear as: 


(s Natural 
{ 
:>0k #'do-nothing 
:>rightmost (A() (error 'cant-happen)) 
}) 


Here #'do-nothing is a stub function that simply returns. Though the 
continuation does nothing, the step function has done something. It stepped 
the natural-number-generator, so the next time we read the machine we 
will see a one larger number. 


There is no end to the set of Natural Numbers. so should we ever call the 
continuation for stepping from rightmost, there is a bug in Natural, and we 
throw an error. This sort of error is of a different nature than that of an end 
case. Unlike an end case, such a control flow change would be unexpected 
and of unknown origin. We have no obvious recourse that would be more 
meaningful than the thrown error. In general it would be unusual code that 
knows what to do in a presence of program bugs. Such code would be 'fault 
tolerant’. 
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Note that continuations are functions, and the >rightmost continuation 
takes no arguments, so the error call must be wrapped in a lambda so as to 
give it an argument. 


There is already a cant-happen function in the library, so instead of the 
lambda expression that throws an error we could have passed in #'cant- 
happen. 


Like s above, many of the tape machine functions are single letters. This 
facilitates stringing function names together to create compound access 
functions, in analogy to lisp's car and cdr access language (cadr, caddr, etc.). 
The access language is then extended into a general out of band signaling 
pattern matching language that works on arbitrarily long data types. See the 
chapter on the TM Access Language. 


Here is a simple program that makes use of a tape machine: 


(let ( 

(a-tape-machine (mk '‘list-tm {1 2 3})) 
) 

(print (r a-tape-machine)) 

(s a-tape-machine) 

(print (r a-tape-machine)) 

(s a-tape-machine) 

(print (r a-tape-machine)) 

(s a-tape-machine) 


) 


When run it produces: 


Zw |S 


IL 


mk accepts a tape machine type and an initializer then returns an 
initialized instance. In this case the leftmost cell will have a'1' in it. Its 
right neighbor will have '2' in it. The rightmost cell will have '3' in it. 


ris a function that accepts a tape machine and returns the instance indicated 
by the head. w does the inverse operation. It accepts a tape machine and an 


object, and then writes the object into the indicated cell. Hence, the first 
(print (r a-tape-machine)) will print the number 1. 


s moves the tape head so that it lands on the right neighbor cell. The second 
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(print (r a-tape-machine)) will print the number 2. Etc. Note we do not 
provide any continuation function arguments to s. They are optional, and the 
defaults are (be t) and (be @). That is why at the final step, when stepping 
fails, the program returns NIL. (The printer is not aware of our alias Ø, so it 
prints NIL.) 


Tape-Machine Primitive Interface 


By interface I mean the set of function class declarations and the functions 
written in terms of the declarations. These latter functions are said to be 
"generic’. 


tm-type. lisp 
tape-machine is the most general type in the TM Library. 
(def-type tape-machine () ()) 


No instance will be made of tape-machine. Rather tape-machine has 
been defined only as a basis for specialization. Functions that have 
parameters with the type tape-machine will instead be passed instances of 
specialized types. Once such an instance has been passed into a function, the 
only thing the function can do with it is to pass it to other functions without 
accessing any slots. This is because tape-machine does not have any slots. 
This turns out to be useful for recognizing generic functions. 


We discuss some implementations in more detail in a later chapter, 
Implementations. Implementations are created by specializing the tape- 
machine through the addition of slots that hold data. Here is the type 
definition for the singly linked list implementation. tape is a reference to the 
list, and head indicates a node in the list. Taking a step will move the head to 
the next node. 


(def-type list-tm (tape-machine) 


(head ; locates a cell on the tape 
tinitarg :head 
:accessor head 


(tape ; a sequence of cells, each that may hold an instance 
tinitarg :tape 
raccessor tape 


) 
)) 


Programmers will notice that such a tape machine resembles an iterator. The 
head is state for the iterator, such as a pointer to a member in the container 
(to a cell holding an instance), and tape is a pointer to the container. Such an 
iterator need not be given a parameter naming the container, as it already 
has a reference to it. 
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Linked lists in Lisp refer to their data by reference, so any instance, 
independent of its type, may be found on the tape for this machine. 


In our morphism between the mathematical abstraction ofa Turing Machine, 
and our TM Library, type is placed into correspondence with the Turing 
Machine symbol, and instances of type are placed into correspondence with 
instances of said symbols. These should not be confused with the syntactical 
element of the Lisp language that are also called symbols. Typically it is clear 
from the context which symbol definition is intended. 


The calling program plays the role of the Turing Machine state controller. The 
calling program sends commands, or statements, by invoking tape- 
machine type class member functions. 


tm-mk.lisp 


For tape-machine specializations with implementations we can create new 
instances using the function mk: 


(defun mk (tm-class keyed-parms &optional >) 
(let( 
(instance (make-instance tm-class)) 


) 


(init instance keyed-parms >) 


)) 


mk has three parameters. The first parameter, tm-class, may be bound to a 
Lisp symbol, where this symbol is a type name, and this type is a tape 
machine specialization with an implementation. Later we will see that list- 
solo-tm is such a specialization with an implementation. Hence, an example 
legal argument to pass in is 'list-solo-tm. 


The second parameter, keyed-parms, may be bound to initialization data 
for the new machine. We might, for example give this argument a value 
of :tape (list 1 2 3), or equivalently, :tape {1 2 3}. For the solo-tape 
machine, this initializes the first three tape cells to the numerical instances 
1, 2 and 3. 


There are three common continuations, though specialized versions of tape 
machines can define others: 


(destructuring-bind 
(&key 
(ok #'echo) 
(> fail (AQ)(error 'bad-init-value))) 
(>no-alloc #'alloc-fail) 
&allow-other-keys 


) 


Exactly one of the continuations will be called as the last thing that mk does. 
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ok is called and passed the newly made instance if all goes well. -fail is 
called if there is a semantic problem initializing the instance. And >no-alloc 
is called if there is not enough memory for creating a new instance. Note each 
of these continuations is set to a default function. Hence, when no 
continuations are specified we get old fashioned behavior, where we make a 
call to mk and it returns a value, or instead throws an error. 


You will notice that mk has been declared with defun, and not defun-typed. 
As such, in Common Lisp, we cannot have multiple versions and then select 
among them depending on the type of the argument. However, mk creates 
an instance and then calls init. and init is defined with defun-typed, so we 
may have different versions of init for the various tape-machine 
implementations. 


(def-function-class init (instance init-value &optional —>)) 


(defun-typed init 
( 
(tm tape-machine) 
(keyed-parms cons) 
&optional > 
) 
(destructuring-bind 
(&key 
(0k #'echo) 
(> fail (AQ (error 'bad-init-value))) 
(>no-alloc #'alloc-fail) 
&allow-other-keys 


) 


= 
(destructuring-bind 
(&key tape &allow-other-keys) keyed-parms 
(cond 
((A tape (typep tape 'sequence)) 
(cue-leftmost tm) 
(w tm (elt tape 0)) 


(loop 
for item in (subseq tape 1) do 


(as tm item {:>ok #'do-nothing :->no-alloc —no-alloc}) 


) 


[>ok tm] 


) 
(tape [>fail]) 
(t [20k tm]) 
)))) 


The def-function-class does little more than provide an argument count. 
We can't even declare defaults for optional arguments. So here in the first 
line we have a declaration for a class of functions called init where each 
function in the class will take two arguments followed by two optional 
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arguments, perhaps followed by more arguments. The list that will capture 
the continuation arguments is a parameter called '>". 


Just below the function class declaration is a definition for an init function. 
This init function will be called if no other init function has a more 
specialized tape-machine class specifier. tape-machine has no 
implementation, so this init version is generic. 


One can also find a shallow copy function in the tm-mk.lisp file. When given 
a tm-machine, it returns another tm machine that has a new head and a 
new tape. However, the tape references for both machines will point at the 
same instances. Chances are good that the entanglement copies are a better 
choice when one desires to have copies that share references. 


tm-decl-only.lisp 


The tm-decl-only.lisp file contains function class declarations. 
Specializations must provide: 


(def-function-class r (tm &optional >)) 
(def-function-class esr (tm &optional >)) 


(def-function-class w (tm instance &optional —)) 
(def-function-class esw (tm instance &optional >)) 


(def-function-class rf (tm &optional >)) 
(def-function-class esril (tm &optional >)) 


(def-function-class wi (tm instance &optional >)) 
(def-function-class eswiD (tm instance &optional >)) 


(def-function-class cue-leftmost (tm &optional —)) 


(def-function-class s (tm &optional —)) 
(def-function-class a (tm instance &optional —)) 
(def-function-class on-leftmost (tm &optional —)) 
(def-function-class on-rightmost (tm &optional —>)) 


(def-function-class tape-length-is-one (tm &optional —>)) 
(def-function-class tape-length-is-two (tm &optional —)) 


These are pretty much what one would expect given the lead up to this point. 
r, w, and s, are read, write, and step. esr and esw perform a read or write 
on the right neighbor (They were described in the section, More about 
commands.) TM commands either operate on the leftmost of the tape, the 
rightmost of the tape, the cell under the head, or on an area to the right of 
the head. Commands that operate on the leftmost cell have a D suffix on their 
names. So for example, rl , reads the leftmost cell. The 'operate on the area 
to the right of the head' type of operators came into being when we 
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supported an expanding tape. The cascading implications suggested it was 
best for esr and esw to be primitive. 


rand w typically do not require continuations, as the head is always on some 
cell, and in the presence of our contract with the programmer that he or she 
always do a write on a given cell before a read on that same cell, there can be 
no error condition associated with these operations. In contrast, s has a 
rightmost function that is invoked when one attempts to step right from 
rightmost. If rightmost is not called, then ok is called. Note it always 
works this way, that continuations are exhaustive of all cases and that at least 
one of the continuations will always be called as the last thing that a function 
does. Though, continuation lists are not fixed, functions that operate on 
specializations might have additional continuations than the those that 
operate on the corresponding more general types. 


We cannot see which continuations are implemented by looking at the TM 
interface function declarations. The only thing we will see is the continuation 
list parameter, ''. To see which continuations are implemented we must 
look at the definitions. Typically there will be a base list of legal 
continuations available for the most general type, with possibly more 
continuations available for functions defined against more specific types. 


Here are the functions in the 'tm-list class: 
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(defun-typed r ((tm list-tm) &optional =>) 
(destructuring-bind 
(&key 
(70k #'echo) 
) 


(defun-typed esr ((tm list-tm) &optional >) 
(destructuring-bind 
(&key 
(70k #'echo) 
(rightmost (A()(error 'step-from-rightmost) )) 
&allow-other-keys 


) 


(defun-typed w ((tm list-tm) instance &optional >) 
(destructuring-bind 
(&key 
(0k (be t)) 
&allow-other-keys 


) 


(defun-typed esw ((tm list-tm) instance &optional >) 


(destructuring-bind 
(&key 
(ok (be t)) 
(rightmost (be @)) 
&allow-other-keys 


) 
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(defun-typed ri ((tm list-tm) &optional >) 
(destructuring-bind 
(&key 
(70k #'echo) 
) 


(defun-typed esr ((tm list-tm) &optional >) 
(destructuring-bind 
(&key 
(70k #'echo) 
(rightmost (A()(error 'step-from-rightmost))) 
&allow-other-keys 


) 


(defun-typed wl ((tm list-tm) instance &optional >) 
(destructuring-bind 
(&key 
(ok (be t)) 
&allow-other-keys 


) 


(defun-typed esw ((tm list-tm) instance &optional >) 
(destructuring-bind 
(&key 
(ok (be t)) 
(rightmost (be @)) 
&allow-other-keys 


) 
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(defun-typed on-leftmost ((tm list-tm) &optional >) 
(destructuring-bind 
(&key 
(>t (be t)) 
(>Ø (be @)) 
&allow-other-keys 


) 


(defun-typed on-rightmost ((tm list-tm) &optional >) 
(destructuring-bind 
(&key 
(>t (be t)) 
(>Ø (be ø)) 
&allow-other-keys 


) 


(defun-typed tape-length-is-one ((tm list-tm) &optional =>) 
(destructuring-bind 
(&key 
(>t (be t)) 
(>Ø (be @)) 
&allow-other-keys 


) 


(defun-typed tape-length-is-two ((tm list-tm) &optional >) 
(destructuring-bind 


(&key 
(>t (be t)) 
(>Ø (be @)) 
&allow-other-keys 
) 
=> 
(if 
(A (cdr (tape tm)) (cddr (tape tm))) 
[>t] [>] 
)) 


on-leftmost and on-rightmost are Boolean functions of convenience. on- 
leftmost takes the true continuation when the head is on the leftmost cell, 
otherwise it takes the false continuation. on-rightmost is follows the true 
continuation when the head is on the rightmost cell, otherwise it takes the 


false continuation. 
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(on-leftmost tm 
{ 
:=>t (AQ) code to be executed when on-leftmost is true) 
:@ (A() code to be executed when on-leftmost is false) 


}) 


If one wants conventional behavior from these tests, then use the be 
functions: 


(on-leftmost tm {:>t (be t) : >Ø (be o)}) 
This call will return t or ọ. Actually, these are already the defaults. So 
(on-leftmost tm) 


will return t or ọ. 


tm-generic.lisp 


This file contains more declarations for function classes on the tm-machine 
interface. However, unlike for tm-decl-only.lisp, the file also has generic 
function definitions. 


(def-function-class cue-rightmost (tm &optional >) 
(:documentation 
"Cue tm's head to the rightmost cell." 


)) 


(defun-typed cue-rightmost ((tm tape-machine) &optional >) 
(declare (ignore >)) 
(labels( 
(work() (s tm {:-ok #'work :—rightmost (be t)})) 
) 


(work) 
)) 


This default implementation for cue-rightmost might not be the fastest 
one, but it will always work. A programmer is free to implement a faster 
version for specialized argument types. In general we need not implement 
functions in these classes, but are free to do so if we see some advantage in 
it. Here are more of the tm-generic.lisp function classes and function 
definitions: 
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(def-function-class as (tm instance &optional >) 
(:documentation 
"Like #'a, but tm is stepped to the new cell 


©) 
(defun-typed as 
( 


(tm tape-machine) 

instance 

&optional > 

) 

(destructuring-bind 

(&key 
(0k (be t)) 
(>no-alloc #'alloc-fail) 
&allow-other-keys 


) 


> 
(a tm instance 


:>0k (A()(s tm {:30k ok :>rightmost #'cant-happen})) 
:>no-alloc >no-alloc 
}) 

)) 


(def-function-class a&hM (tm instance &optional >) 
(:documentation 
"#'a with a contract that the head is on rightmost. 


D) 
(defun-typed a&hil 
( 


(tm tape-machine) 
instance 

&optional > 

) 

(a tm instance >) 


) 


(def-function-class as&hM (tm instance &optional >) 
(:documentation 
"#'as with a contract that the head is on rightmost. 


")) 
(defun-typed as&hii 


(tm tape-machine) 
instance 
&optional > 
) 
(as tm instance >) 


) 
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Quantifiers 


I build the quantifiers on top of a self-recursion (looping) function that I call 
‘do’ and represent with the 'C’ symbol. Initially the reason I used this 
Unicode symbol is because all the good names for looping had already been 
taken. However I have come to like my little looping operator. It is too bad 
most programmers won't see it, because, when possible, programmers 
should instead use the quantifiers shown further below. They are easier to 
control. 


(defun © (work) 
(labels( 
(again() (funcall work #'again)) 


(again) 

)) 
The first and only argument passed to work is a function, which when called, 
will cause work to be called again. The return value from C will be the return 
value from the last call to work. 


Actually the library does not implement the definition given above, because 
I found that the compiler was not optimizing out the recursion. As of version 
0.7 the implementation is: 


(defun © (work) 
(let(return-value) 
(tagbody 
again 
(setf return-value (funcall work (A()(go again)))) 


) 


return-value 


)) 
Existential quantification is based on C: 


(defun 3 (tm pred &optional (>t (be t)) (>Ø (be @))) 
(C(A(again)[pred tm >t (AQ)(s tm {:o0k again :>rightmost >@}))])) 
) 


When pred is true, the quantifier exits with >t. Otherwise pred continues 
on to step the machine and to call pred again. Should the step go right of 
rightmost, the quantifier exits with > Ø. 


q stands for 'there exists' and it has multiple intuitive interpretations. One 
can think of it as a short circuiting logical OR. It can be thought of as ‘loop 
until’. It is also a linear search for an instance with a particular property. For 
example, Diogenes wanders Greece looking for a good person. He stops his 
search upon finding one. 


We build universal quantification from existential quantification: 
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(defun v (tm pred &optional (>t (be t)) (>Ø (be @))) 
(a tm (A(tm ct cø)[pred tm cø ct]) >Ø >t) 
) 


v stands for 'for all’. This can be thought of as a short circuiting logical AND, 
as loop while, or as a check that all is good. 


Here are two trivial predicates, presented so you can see the general form: 


(defun always-false (tm >t >) 
(declare (ignore tm >t)) 
[>] 
) 


(defun always-true (tm >t >Ø) 
(declare (ignore tm >@)) 
[>t] 
) 


The star suffixed version 4* traverses until the end of the tape and returns a 
pair, the first of the pair being a count of the tests done, and the second being 
the number of tests that were true. 


The star suffix version V* of also traverses to the end of the list. It does not 
return anything. 


The quantifiers begin with the cell the head is on for the tape machine that 
is passed in. Versions with a prefix of 'cIl', which is the cue-leftmost 
command, will first cue to the leftmost of the tape, i.e. c03, CIV, cI3*, 
civ*. 


If one wants to collect results, do so on a tape machine that is available in the 
closure. One would typically use the as command to do so. 


The quantifiers may be used to implement the same functionality as map 
and as while loops. In a later chapter we discuss machines that create ranges 
of numbers. When used with these machines the quantifiers can be used to 
create for loops. Note also, the ensemble machine is used to bind multiple 
machines together so they step as one machine. 
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This function will return true: 


(defun test-3-0 () 
(let*( 
(y {1 2 {3 4} 5}) 
(ytm (mk 'list-tm {:tape y})) 
) 


(A 
(a ytm 
(A(tm >t >28) 
(if 
(A (typep (r tm) 'cons) (eql 3 (car (r tm)))) 
[>t] 
[>] 


))) 
(equal (r ytm) '(3 4)) 
D) 


So will this one: 


(defun test-3-1 () 
(let*( 

(y {1 2 {3 4} 5}) 

(ytm (mk 'list-tm {:tape y})) 
) 

(a ytm 
(A(tm >t >Ø) 

(if 

(A (typep (r tm) 'cons) (eql 3 (car (r tm)))) 
[>t] 
[79] 


)) 
(AQ) (equal (r ytm) '(3 4))) 
#'cant-happen 
))) 


Generators 


A generator is an abstract tape machine that is typically used to generate 
data, such as a sequence of numbers or letters. Generators may be stepped 
and read like a conventional machine, but the value read is that created by 
an internal function, rather than something that was written. 


Recursive 


The recursive machine is given an initial value and a function. After a cue 
leftmost command, reading the machine returns the initial value. Upon each 
step, the provided function is applied to the prior value. The result can be 
read by reading the machine. 
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When invoked, the provided function is provided a value and two 
continuation functions. Should the function successfully compute a result, it 
calls the first continuation and passes it the result. If the function cannot 
compute a result, then it calls the second continuation. This is useful for 
creating finite sequences. 


For example, this program returns true: 


(defun test-recursive-0O () 
(labels( 
(inc-to-10 (i0 c-success c-fail) 
(let( 
(i1 (+ i0 1)) 
) 


Gf (S id, 10) 
[c-fail] 
[c-success i1] 


))) 


) 

(let( 
(tm (mk 'recursive {:initial 1 :f #'inc-to-10})) 
(result ø) 


) 
(v* tm (A(tm) 
(setf result (cons (r tm) result)) 


)) 
(equal result {1098 765432 1}) 
))) 


Here we use labels to locally define the function inc-to-10. When passed a 
value less than or equal to 10, it adds one to it, and calls c-success. 
Otherwise it calls c-fail. Though we are using numbers in this example, we 
may work with instances of any type for which we can define a function for. 


In Lisp a node ina linked list is called a cons cell. Such a cell is pair of values, 
where the first in the pair is called the car, and the second in the pair is called 
the cdr. The car holds an integer that might be interpreted as a number or 
as a reference to a value, while the cdr holds either @ or a reference to the 
next cons cell. These terms have no special meaning outside of this context. 
Hence the function (cons (r tm) result)) creates a list node with the value 
read from tm, and the reference to the next node in the list being the value 
of result. Or in short, it prepends a value to the result list. 


Inside the let, we create a recursive machine while passing an initial value of 
1, and our inc-to-10 function. Then we use a quantifier to run through all 
the instances in the recursive machine. We use cons to repeatedly prepend 
the instances to a list. Finally we check the list. 


When working with vectors it is common to generate short sequences of 
integers. Hence, we have packaged the work of making an increment 
function with a bound and a recursive machine into a single call, mk- 


78/148 


interval. 


(defun test-recursive-3 () 
(let( 
(tm (mk-interval 1 9 2)) 
(v #(8163452709)) 


) 
(v tm (A(tm ct cø) 
if 


(= (elt v (r tm)) (r tm)) 
[ct] 
[cø] 
))) 
)) 


Here we use mk-interval to create a recursive machine that will enumerate 
the integers in an interval extending from 1 to 9. It will count by twos while 
doing so, thus generating the values 1, 3, 5, 7, and 9. We then use these values 
to index an array. elt is the function Lisp uses to index into an array. | 
initialized the array so that the values at 1, 3, 5, 7, and 9, are the same as their 
index. The quantifier then checks that indeed this is true in all cases for tm. 


With TM we use generators and quantifiers rather than for loops, or 
dotimes loops. 


We have also defined a function for making the Natural Numbers, as this was 
used as an example earlier in the book, mk-Natural. 


Here is an interesting, though silly, little program that uses mk-Natural. 


(defun interesting-O () 
(let( 
(i (mk-interval 0 9)) 
(N (mk-Natural)) 
(count-N 0) 
(count-odd 0) 
(count-even 0) 


) 
(v* i (ACi) 
(declare (ignore i)) 
(incf count-N) 
(if (oddp (r N)) (incf count-odd) (incf count-even)) 
(s N) 


)) 
(A 
(= count-odd count-even) 
(# count-N count-odd) 


))) 


The program is interesting in that the interval is protecting the loop so that 
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there will be a return value. It returns true because there are five even and 
five odd numbers encountered. The total numbers traversed is 10, which is 
not the same as the count of odds. The program is a little bit silly, because 
iterating through the interval also produces Natural Numbers, so we didn't 
need N. Here is a more 'pure' use of Natural: 


(defun interesting-1 () 
(let( 
(N (mk-Natural)) 
(count-N 0) 
(count-odd 0) 
(count-even 0) 


) 
(v* N (A(N) 
(incf count-N) 
(if (oddp (r N)) (incf count-odd) (incf count-even)) 
)) 
(= count-N count-odd count-even) 


)) 


This might be a good time to mention that hitting control C twice in Emacs 
interrupts a process. Though this program is not computable, in a fashion it 
is analyzable. When we analyze it, we are surprised to find that there are just 
as many odd, or even, Natural Numbers as there are Natural Numbers in 
total, i.e. that the routine returns true. Apparently, somehow, on the way to 
infinity, the odd and the even counters catch up to the N counter. 


Specializations 


As we mentioned in the More about commands section of the prior chapter, 
by simultaneously supporting multiplexing and a cell delete command we 
have created the possibility of colliding operations, i.e. a collision hazard. We 
can avoid this hazard by leaving out the delete command, or alternatively, by 
not allowing multiplexing. These choices lead to a specialization hierarchy 
for the interface: 


tape-machine 


/ \ 
nd-tape-machine solo-tape-machine 
\ / 


haz-tape-machine 


The nd-tape-machine type class does not have a delete command, but it 
does support multiplexing. When we do not provide a delete cell command 
on the interface, the programmer cannot create an event where the cell 
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under the head gets deleted. Such an interface can have multiplexing without 
hazards. 


It is instructive to go through the code base for the TM Library and take 
notice how much simpler the functions belonging to the nd-tape-machine 
function class are relative to those of other classes that we describe in later 
chapters. Before this tape machine library the more complex design for 
destructive programming, which are encapsulated in machines here in this 
library, were spun ad hoc by programmers, and hence, this added complexity 
directly translated into complexity in the programming task. 


Though non-destructive programming is simpler, it leads to unnecessary 
recopying of data structures just to avoid simple destructive changes such as 
appending or deleting an element in a list. 


Technically the Turing Machine does not have a delete cell command, but we 
can emulate one by moving all the instances on the right-hand side left by 
one. Because the head is on the left-hand side, we do not have symmetry. i.e. 
moving all the instances over one on the left-hand side does not delete the 
cell under the head, though it does delete the right neighbor. 


The solo-tape-machine includes commands for deletion, but not for 
multiplexing. If we never have more than one head (i.e. never more than one 
iterator), then one head cannot be used to delete the cell to the right, and 
wipe out a cell that a second head is on. Consequently there is no collision 
hazard. 


Without multiplexing we cannot make copies of the head state (i.e. have an 
additional machine head), and this prevents us from writing some library 
functions that transparently to the caller operate on tape machines. 


Again it is interesting to notice how much complexity is avoided compared 
to classes we describe in later chapters. However, notice all the functions the 
nd-tm-decl-only and nd-tm-decl-generic source files that are not 
available in the solo-tape-machine function class. These even include a 
simple print function, because the print function wants to make an entangled 
copy and cue its head to leftmost. 


So at the first level of specialization there is an either-or situation, the 
programmer can have multiplexing, but not deletion, with nd-tape- 
machine; or have deletion but not multiplexing with solo-tape-machine. 
But what if we do an evil thing, and diamond inherit both of these types into 
a third type? Then we get tm-haz where ‘haz’ is short for hazardous. With 
tm-haz it is possible to break the machine by calling entangle to get a 
second head, and then to use that second head to delete the cell the first head 
is on. Hence, tm-haz machines may only be used without potential for errors 
when either a) there is a proof that such a collision will never happen b) they 
are controlled from a second level machine such as the ea-tm or its 
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specializations, as discussed in the chapter on Second Level Machines. 


nd — multiplexing but non-destructive 


The non-destructive programming model interface is defined in the 'nd-tm' 
files. Here 'nd' stands for non-destructive. As mentioned, there are no 
commands on the interface to delete (deallocation) a cell from the tape. 


(def-type nd-tape-machine (tape-machine)()) 


Non-destructive machines support having multiple heads on the same tape 
without hazards, because there will never be a time when one head attempts 
to delete the cell that another head is accessing. To place a second head ona 
tape we make a new machine that shares another machine's tape. When two 
machines share the same tape, we say they are entangled. To create an 
entangled machine call the function entangle. 


(def-function-class entangle (tm-orig &optional >)) 


We pick up a couple of new predicates, one to test if two machines are 
entangled, and another to test if two entangled machines have their heads 
on the same cell. 


(def-function-class entangled (tm0 tm1 &optional >)) 
(def-function-class heads-on-same-cell (tmO tm1 &optional —>)) 


Given the ability to make entangled copies we can implement some new 
functions. 


(defun-typed s# 
( 


(tmO nd-tape-machine) 
(tmi nd-tape-machine) 
&optional > 


(destructuring-bind 
(&key 
(ok (be t)) 
(rightmost (be @)) 
(bound (be @)) 
&allow-other-keys 


) 


=> 
(heads-on-same-cell tmO tm1 


K 


:>t >bound 
:>Ø (A()(s tm0 {:>0k ok :>rightmost >rightmost})) 
»)) 
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s+ is similar to s, but will take a bound continuation when one attempts to 
step beyond the bound set by tm1. 


(defun-typed alll 
( 


(tm nd-tape-machine) 
instance 
&optional > 


(destructuring-bind 
(&key 
(ok (be t)) 
(>no-alloc #'alloc-fail) 
&allow-other-keys 


) 


=> 
(let( 
(tm1 (entangle tm)) 


(cŒ tm1) 
(a tm1 instance {:-0k >ok :->no-alloc >no-alloc}) 


))) 


The function al adds a new cell to the rightmost of the tape. Inside this 
function an entangled copy is made, the entangled copy is then cued to the 
rightmost, a cell is appended, and then the entangled copy is abandoned. The 
caller's state is not modified. 
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We also pick up some quantified functions, such as esnr: 


(defun-typed esnr 
( 


(tm nd-tape-machine) 
index 
&optional > 


) 


(destructuring-bind 
(&key 
(20k #'echo) 
(rightmost 
(A(index) 
(declare (ignore index)) 
(error 'step-from-rightmost) 


)) 


&allow-other-keys 


) 


=> 
(let( 
(tmi (entangle tm)) 


(sn tmi index 


:30k (AQ[ok (r tm1)]) 
:> rightmost (A(n)[>rightmost n]) 


} 
)))) 


Here esnr stands for ‘entangled copy, step n times, and read’. This function 
allows a program to read an instance n cells away from the current head 
position, without moving the head of the machine passed in. There are also 
analogous write and append versions of this function. 


solo - destructive operations allowed but uniplex 
(def-type solo-tape-machine (tape-machine)()) 


The 'solo' machine does not have an entangle command on its interface, so 
there will be exactly one head on a given tape. Consequently its controller 
will be a uniplex machine. Without multiple heads there can be no hazardous 
situations where deleting a cell would break the machine. 


However, without the entangled copy function it is not possible to create 
temporary machines that go off and do some work without perturbing the 
head position of the original machine.2° Consequently we can't have 


2°On implementations that supports left stepping, it would be possible to step the head 
and count the steps, to perform an operation, and then step the head the opposite 
direction using the same count. However, the problem doesn't go away, as if the 
cell the head was originally on gets deleted, we cannot return the head to that cell. 
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functions like afl which internally does an entanglement copy so as to not 
perturb the machine accepted as an argument. 


As for the nd-tm machines, solo machines add no overhead. They only add 
functionality. 


We gain these primitives: 


(def-function-class al (tm instance &optional >)) 
(def-function-class d (tm &optional spill —)) 
(def-function-class di (tm &optional spill —)) 


all appends a new cell to the leftmost of the tape. The new cell is initialized 
to instance. d deallocates (deletes) a cell, and dH deletes the cell at the 
leftmost of the tape. The spill machine is optional, if it is present, the instance 
spilled from the deallocated cell is appended to it. Upon success the delete 
commands will call the ok continuation with a single parameter of the 
spilled cell. The ok continuation defaults to #'echo. There is a bit of an 
awkward situation in that to provide continuations we must also provide a 
spill machine, as optional parameters occur in order. In such a situation, if 
there is not a spill machine, then set spill to Ø. 


Implementations 


Implementation specialization tree 


Our first implementation uses a singly linked list for a tape. A singly linked 
list has the disadvantage that stepping left is not possible. However, lisp 
programmers have a lot of experience in working around this limitation. The 
primitive functions without implementations appear in a specialization 
hierarchy, so we add our specialization as leaves in that hierarchy. We will 
have an analogous hierarchy for each implementation. It ends up looking like 
this: 
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[tape-machine] 


list-tm 
AN 

/ \ 

[nd-tape-machine] / \ [solo-tape-machine] 
\ / \ / 
list-nd-tm list-solo-tm 
\ / 
\ / 
[haz-tape-machine] \ if 
ae / 


list-haz-tm 


The machines shown in square brackets are the interface definitions we have 
already talked about. The other machines on this diagram are specialized to 
provide a single direction, right going only, linked list implementations. 


File Name and Function Name Convention 


The code discussed in this chapter is organized into directories. The src-list 
directory holds both the interface definition code and the single direction 
and bi-direction list implementations. 


[src- | test- implementation 
implementation:: list ... 


Each source directory starts literally with 'src-' and then the implementation 
name follows. Each source directory is paired with a test directory. The test 
directory holds files with the same names as those in the source directory, 
and the contents of those files are tests for the functions found in the 
corresponding source files. Additional test files will also be in the directory. 


The files are pulled together in the tm.asd file. The tm package uses asdf for 
build management. 


TM source file names have four parts. If a part is empty, then it does not 
appear in the name. 


[implementation][hedge|tm-category.lisp 
implementation:: e | list- | array- | .... 

hedge:: e | nd- | solo- | ea- | ts- 

category:: describes types of functions held in the file 


For example, tm-deflisp holds def type files. This file holds general tape- 
machine source code, that which has no implementation, and does not have 
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either delete cell, nor multiplexing. As another example, list-solo-tm- 
primitives.lisp. is a file that holds functions for a list implementation that uses 
the solo hedge to avoid the collision hazard. The file holds primitive 
functions. 


Types without implementations have tape machine spelled out, tape- 
machine, nd-tape-machine, etc. In implementations and for file names, 
tape machine is always abbreviated as tm. 


TM test file names are of the form: 
test-name-series 
name:: name of the function being tested or a descriptive name 
series:: a number, typically sequential starting from zero. 


Test functions return t when they pass. If there is an exception or the test 
returns something else, it has failed. To add a test to the regression suit, use 
the following command: 


(test-hook function-name) 


Where function-name is the name of the function being added to the 
regression. To run the regression type: 


> (test-all) 


list-tm types 


(def-type list-tm (tape-machine)())) 

(def-type list-nd-tm (nd-tape-machine list-tm)())) 

(def-type list-solo-tm (solo-tape-machine list-tm)()) 

(def-type list-ea-tm (ea-tape-machine list-nd-tm list-solo-tm)()) 


We may make instances of these types by calling mk. 


Here are some examples. All of our test functions return true, t. The tests are 
loaded with the library, and can be run with the command (test-all). 
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(defun test-heads-on-same-cell-O () 
(let*( 
(tm0 (mk 'list-nd-tm {7 2 -3})) 
(tmi (make-instance 'list-nd-tm)) 


) 

(init tm1 tm0) ; entangles tm1 with tmO 
(A 

(s tm0) 

(~ (heads-on-same-cell tm0O tm1)) 

(s tm1) 

(heads-on-same-cell tmO tm1) 

(s tm1) 

(~ (heads-on-same-cell tm0O tm1)) 


))) 


(test-hook test-heads-on-same-cell-0) 


(defun test-dn-0 () 
(let*( 
(tm0 (mk 'list-solo-tm {1 2 3 4 5})) 
(tmi (mk 'list-solo-tm {-100})) 


) 
(A 
(s tm0) 
(dn tm0 2 tm1) 
(equal (tape tm0) {1 2 5}) 
(equal (tape tm1) {-100 3 4}) 
(on-rightmost tm1) 
(= (dn tm0 10 ø (be -1) #'echo (be -2)) 9) 


))) 
(test-hook test-dn-0) 


(defun test-ea-all-0 () 
(let*( 
(tm0 (mk 'list-ea-tm {1 2 3})) 


) 
(with-mk-entangled tm0 
(A(tm1) 

(s tm0) 

(w tm0 22) 

(al tm0 7) 

(A 
(equal (tape tm0) {7 1 22 3}) 
(equal (tape tm1) {7 1 22 3}) 
(= (on-leftmost tm0)) 
(= (on-leftmost tm1)) 
(eql (r tm0) 22) 
(eql (r tm1) 1) 
))))) 

(test-hook test-ea-all-0) 
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Initialization and the First-Rest Pattern 


A tape machine can only be created after there is a first item to place in it. 
Consequently code that produces data that is placed in a tape machine will 
often have separate first case and recursive case. 


(get-first 
{:0k (A(tm) 
(get-rest tm) 
)}) 


In this example, get-first gets a first data item. If that item is not the last 
item, and there was not an error, get-first creates a tape machine and then 
calls a continuation with a tape machine initialized to this data item. get- 
rest then gets the rest of the data items but does not have to create a 
machine. We call this the 'get-first/get-rest' pattern. 


When programming with loops this pattern manifests as ‘loop priming’. 
Accordingly work is done before entering a while or for loop, where this 
priming work is a variation of the work done inside the loop. Here get-first 
is the priming part, and get-rest is the loop and its contents part. 


Suppose we want to avoid having two functions, but instead just want a 
single get function that plays either or both roles. Consider the case where 
get takes a tm parameter. We can pass in Ø, so that get knows it has to create 
a new machine. 


(get tm 
(if (not tm) 
(get-first) 
(progn 
(get-rest tm) 
tm 


))) 


When we look inside such a get function, we will find a test on tm to see if it 
is Ø. The true path will execute code which we may as well call get-first, and 
the false branch will execute code which we may as well call get-rest. We 
didn't avoid the get-first/get-rest pattern, rather we just encapsulated it. 


However, we have a problem with get. We cannot create a tape machine in a 
function and pass it back out though the parameter list, because the tm in 
the function is a local variable, and we do not have a reference to the tm in 
the caller. Consequently we cannot initialize the caller's machine. We can add 
to the tape when a tm is passed in, but we can't create the first cell. 
Consequently get we will always have to return tm and we end up with this 
pattern for using get: 
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(let ((tm @)) 
(setf tm (get tm)) 
) 


So we find that the get with a flag approach adds a redundant test each time 
it is called, unnecessarily returns tm, and must redundantly assign to the 
outer tm on each call. It appears to be better to just use the get-first/get- 
rest pattern directly. 


Emptiness 


Ifa machine could hold any number of elements, including being empty, then 
the get function discussed in the prior section would not require the return 
value and outer assignment. 


When we were defining a conventional Turing Machine, we were led to make 
the empty-symbol a distinct property of the Turing Machine rather than 
including it in the alphabet. Also in that section we discussed the possibility 
of not having a distinct empty-symbol. We made the empty-symbol distinct 
in our definition because it was special from a computation point of view; 
namely, emptiness existed on the infinite tape, and thus could not have been 
put there by a Turing Machine computation. 


When we discussed addresses in section, Addresses and Cells, we defined 
the length of an area to be one greater than the differences in the addresses. 
This definition has the nice property that an area consisting of a single cell 
has a length of 1. We then used analysis to derive a higher level construct 
where an area of 0 size was the same thing as saying the area did not exist. 
We also found that, though a zero length area does not exist, if can still have 
a location. One might think of it as a locator for growing the area in the future 
should we chose to do so, or as a marker as to where the area used to be. 


Just as there is no way to compute and make an empty tape, there is also no 
way to compute that a tape is really empty. In analogy to defining an empty- 
symbol, and an area of zero length, we will need a higher level construct (i.e. 
belonging to a higher order analysis) so that we can keep track of an 
emptiness attribute for the machine. 


Now consider our Turing Machine variation where our tape grows. Is this 
variation helpful here? What is the minimum sized tape? We never really said 
how many cells we start with. It should have been apparent that the initial 
tape must have at least one cell as the head was said to always be over a cell. 
Perhaps we should have modified our ‘always write before read rule’, and 
instead made it something like 'always append a first cell before the first 
write, and always write before read’. We would have had to have added a flag 
to signal that the tape had not been created yet, so as to know to append the 
first cell. Thus, again, we have discovered we need additional information. 
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The Turing Machine is a mathematical object, whereas the tape-machine is 
a computer programming type, and we deal with instances of this type. 
However, properties of the Turing Machine have implications for our 
implementation. And accordingly we know that we will need additional state 
information to create a tape machine which can be empty. 


Consider our append function. It is passed in a tape machine instance, say 
tm-0, and an instance of something to be appended, say x. If tm-0 has never 
been written, then the append function should create a leftmost cell and 
write x. This is appending to an empty active area. I.e. an active area of zero 
length. However, in contrast, if we pass in a machine with the head on the 
leftmost cell, and that cell has already been written, then our append 
function should create a new right neighbor for the leftmost cell, and write x 
into that. Here it is appending to an active area of length 1. Creating a new 
leftmost cell, and appending to a leftmost cell are fundamentally different 
operations, and they are being distinguished by the length of the active area. 
However, to keep track of the length of an active area, while including the 
possibility for zero length, we again find that we need an external construct. 


Our experience with emptiness thus far, and the theoretical results of 
chapter where we derived our Turing Machine variation, leads to the 
conclusion that such a machine that can appear to be empty will simply be 
repackaging the flag contemplated in the prior section during the discussion 
of the get-first / get-rest pattern. That is to say we will just be moving the 
logic around, and the pattern will remain intact. 


Signaling Emptiness with a @ tape slot 


A machine in the empty state would have no cells on its tape. In the specific 
case of the list-tm machine, which employs a Lisp list to implement the tape, 
we are tempted to simply use a null pointer in the tape slot to signal that the 
machine is empty. The cost is pretty low, in the case of a null pointer we 
remove one possible encoding for a pointer value that otherwise might be 
used to access memory. Actually, in Lisp we have typed pointers, so we could 
use a reference to any instance that is not of cons cell type to signal 
emptiness. 


Signaling emptiness by using a special instance in Lisp, or the use of a null 
pointer, is practical. However, we have another issue with the special tape 
slot (tape field) value approach - we do not require that all machines have a 
tape slot, and when a machine has a tape slot, we do not dictate how it is 
used. In other words, we are specifying an architectural interface, not an 
implementation. Consequently the best we can do is to define a new function 
class as part of the interface and leave it to those who implement the 
functionality behind the interface to find a way to express emptiness for any 
tape machine type they happen to make. However, it would be much better 
to have an elegant solution at the architectural level instead of leaving it as 
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an open question and relying upon ad hoc solutions. 


Signaling Emptiness with a @ machine reference 


Suppose we back this up one level. Instead of using @ as a tape reference 
when the machine is empty, suppose instead we use a @ reference to a 
machine to mean the machine is empty. This would overload ¢ to mean both 
that there is no tape machine, or that the tape machine is empty. We might 
generalize and say that @ means no data is available, no matter what the 
reason is. 


If we were to do this, argument passing would be a challenge. We discussed 
this issue in the prior section about the get-first/get-rest pattern. Let’s 
look at this again a little deeper. 
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(defun c7-0 (x) (setf x (cons 7 x))) 
(defun c7-1 (x) (cons 7 x)) 

(defun c7-2 (x) (append x {7})) 
(defun c7-3 (x) (nconc x {7})) 


(let( 
(a0-0 Ø) 
(a0-1 ø) 
(a0-2 Ø) 
(a0-3 Ø) 
(at oar) 
(ai=1 Gar) 
(at-2 {'a}) 
(a1-3 {'a}) 
) 


(c7-0 a0-0) 
(c7-1 a0-1) 
(c7-2 a0-2) 
(c7-3 a0-3) 


(c7-0 a1-0) 
(c7-1 a1-1) 
(c7-2 a1-2) 
(c7-3 al-3) 


(print "aO-0:")(princ a0-0) 
(print "aO-1:")(princ a0-1) 
(print "aO-2:")(princ a0-2) 
(print "aO-3:")(princ a0-3) 


(print "al-0:")(princ a1-0) 
(print “al-1;")( prince al-1) 
(print “ai-2:")( prince al-2) 
(print "a1-3:")(princ a1-3) 
) 


Here our intention is that function C7 will add the numerical instance 7 toa 
list. We have created four versions of C7. In the first attempt we try to 
prepend a value by creating a cons cell and assigning that to the passed in 
argument. The second uses the cons without the setf. The third call attempts 
to append the 7 to the list using Lisp's append function. The fourth one 
appends the 7 using Lisp's nconc function. All four accept x, the list to be 
modified. We then call our four functions with @ so C7 will be initializing the 
list, then again with a symbol, 'a, so that c7 will be modifying the list. We 
then print the results: 
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"a0-0:" NIL 
"a0-1:" NIL 
"a0-2:" NIL 
"a0-3:" NIL 
"a1-0:" (A) 
tal L(A) 
ni 2A) 
talo (A 7) 


None of the attempts at initialization worked. Only the last attempt at 
modification worked. The reason the function cannot initialize the list 
passed in is that it has no reference to the list. x only holds the value @. There 
is no location information in Ø, and it is impossible to initialize something 
when you don't know where it is. 


Now consider the case where we try to modify the list, initially we have a 
reference to a list in one of the 'al variables’, a1-0, a1-1, ... etc. When we 
pass this reference into C7, the reference is copied. Then x holds a reference 
to the same instance that the al variable referenced. 


In the first function, c7-0, when we setf on x it modifies x, and so x no longer 
points to the original instance. However the al variable is still pointing to 
the original instance. 


before 
(setf x (cons 7 x))) 


al Hi 
a ý 
X 
al 
i after 
e” A (setf x (cons 7 x))) 
x — 7 
From the caller's point of view, nothing has changed. In the second function, 
c7-1, cons returns a value, but nothing is done with it. 


In the third attempt to modify the list, c7-2, append is called with x and 7. 
However, append, like cons, is non-destructive. It simply returns a new list 
that is a shallow copy with the new value, 7, appended. This is returned, but 
we don't use it, so nothing happens. 


In the fourth and successful attempt, C7-3, x points to the same instance as 
the al variable in the caller. This is the first element in the list. We do not try 
to change this. Instead we go to that first element, and we modify it. nconc 
is destructive (i.e. it does not copy the list like append does). Afterward both 
the al variable and x are pointing to the same modified first element, and 
that element points to the new value. 
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before 
(nconc x {7}) 


ps 
oa 
m 
ne 


after 
(nconc x {7}) 


Thus using @ to indicate emptiness requires either 1) that we add some sort 
of level of indirection so that callers and functions are referencing the same 
instance, and then the common instance is modified, or 2) the caller 
somehow gets a reference to the original variable in the caller. 


Programmers using the C language often implement option 2 by passing in a 
pointer to the al variable (or whatever variable is being used) instead of 
passing in the variable value directly. In this way the function has a reference 
to the caller’s version of al and can use that reference to set al to Ø. Inside 
of such a function one must dereference the passed in value to see al. As al 
is also areference, one must dereference twice to see the value a1 is pointing 
to. We can do something similar in Lisp by passing the argument 'in a box’. 
Such a box is defined in the src-0/functions. lisp file.21 


Of course, macros do have access to the caller's variables. 


(defmacro c7 (x) `(setf ,x (cons 7 ,x))) 


* (let( 
(a0 Ø) 
a {'a}) 


(c7 a0) 
(c7 al) 
(print "a0:")(princ a0) 
(print "al:")(princ a1) 


) 
aor" (7) 
Nadi (7 A) 


1The box implementation is taken from dmitry_vk's comment on Stack Exchange. 


95/148 


As another issue with using a Ø reference to mean empty is that we are 
forced to pay the initialization costs for building the container at the point 
the first item is added, rather than incurring that cost at a place of our 
choosing. For a Lisp list, the container overhead correlates directly to the 
number of cells, and a zero cell list has zero overhead, so distinguishing 
between an empty list and having no list is at best moot. In contrast, our tape 
machines have top level structure that exists even when the tape is empty. 
Depending on the tape machine implementation, this top level structure 
might be complex, and might take some time to build. 


Also when we use a Ø reference to mean empty, we cannot hold the overhead 
state from a prior emptied machine to be reused when a new instance is 
added. We are forced to redo the initialization each time the machine 
transitions from empty to not empty. 


As a third issue, if we want to set the reference to a container to Ø to mean 
the container is empty, then we must set all the references to the container 
to Ø, not just the one we would normally be passed as an argument. The 
reference to a reference approach, or the padding cell approach, works well 
for solving this problem, as all the secondary references will point to a single 
container reference, and only that single container reference need be 
modified. 


Signaling Emptiness using a padding cell 


We used the 'option 1' mentioned in the prior section, that of using a padding 
cell, to facilitate signaling emptiness in the version 0.1 of the Lisp TM Library. 
This technique can still be used of course. The programmer need only ignore 
the leftmost cell in machines. If a machine has only a leftmost cell, then by 
this technique the machine is interpreted to be empty. The need for this 
appears most commonly when a machine is passed into a function so as to 
capture a result, because we must first create an empty machine (no result 
yet) and then pass it into the function. The padding cell approach works 
nicely with our level 1 machines because they already have a requirement of 
having at least one cell. 


Here is a version of our c7 program that uses a padding cell. 
(defun c7 (x) (setf (cdr x) (cons 7 (cdr x)))) 


This sets the reference to the cell after the padding cell, to a node created 
with cons. That node has the value 7 and then points to the rest of the list. 
The only difference from our prior code is that x has been replaced with (cdr 
x). This is done to skip the padding cell. This produces the desired behavior. 
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(let( 
(a0 {'padding}) 
(al {'padding 'a}) 
) 


(c7 a0) 
(c7 al) 
(print "aO:")(princ (cdr a0)) 
(print "a1:")(princ (cdr a1)) 


) 
aor" (7) 
Nadie! (7 A) 


Note we also use cdr in the print to skip the padding cell. Had we not done 
this, the lists would have printed with a first entry of 'padding. 


Should we delete all the cells in the list, except the padding cell, the list will 
be considered empty. All references to the list are pointing to the padding 
cell, so they don't need to be updated. 


The padding cell approach is related to the reference to a reference. The 
second reference being embedded in the cell. Though we also gain a data 
value which can be used to hold a property list or something similar. 


The padding cell approach is uniform with the structure of the tape when the 
underlying data structure is a list, but will not be so when it is a different 
structure, such as an array or hash table. In those cases we might end up with 
acons cell used for padding that then points to an array or hash table. 


Using a Second Level Machine 


In the next chapter we will discuss second level tape machines which carry 
status, such as being empty, while wrapping a base machine. The base 
machine may be of any type. The second level machine has its own tape 
machine interface, and watches the operations that goes through it while 
passing most of them to the first level base machine. We use subtypes to 
represent state (status), so the cost is partly hidden by CLOS, which must do 
the dispatch work anyway. However, we still have an extra layer of 
dereferencing when the base machine is accessed. 


Second Level Machines 


Status - status-tm 
Rather than having state bits, we let CLOS keep track of the state by keeping 
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multiple specialized types for a status-tm:22 


1. abandoned 

2. empty 

3. parked 

4. active 
An abandoned machine is one that has been especially marked as being left 
for garbage collection. Any operation on such an abandoned machine will 
signal an error. This is for purposes of debugging. An empty machine acts like 
there are no cells on the base machine tape. A parked machine acts like the 
head is not on the tape. Though the head is conceptually not on the tape, thus 
not on any cell, it does have a right neighbor, and that neighbor is the leftmost 
cell of the tape of the base machine. This definition for a parked head gives 
us the ability to use an area that includes the leftmost cell. Note this 
additional structure is maintained by the higher level construct, i.e. the 
status machine, and not by the base machine. The base machine definition 
does not change. 


(def-type status-tm (identity-tr) 


(base ; the machine being managed 
:initarg :base 
:accessor base 


(address ; an integer address locating the head 
:initarg :address 
:accessor address 


(address-rightmost ; address of the rightmost cell 
:initarg :address-rightmost 
:accessor address-rightmost 
) 

)) 


(def-type abandoned (status-tm)()) 
(def-type active (status-tm)()) 
(def-type empty  (status-tm)()) 
(def-type parked (status-tm)()) 


Because we are analyzing the operations performed on the base machine, we 
can easily see the steps and keep an address integer, so we take the 
opportunity to do that. This makes it possible to compare locations between 
status machines independent of the base type, and it speeds up simple 


22'Status' means the same thing as 'state', but the word state is already overloaded due to 
the state machine in the Turing Machine definition. I.e. we are calling state in the 
second level 'status' so as to distinguish it from state in the first level. 
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locations tests and distance calculations. 


The programmer only makes status-tm machines. Though internally a 
machine is always one of the specialized types, abandoned, empty, parked, 
or active. These specialized types have no new slots, rather the type tags are 
just being used to tell CLOS which interface to use. 


(defun test-status-1 () 
(let*( 
(tO (mk ‘list-solo-tm {:tape {1 2 3}})) 
(t1 (mk 'status-tm {:base t0})) 


) 
(A 

(park t1) 
(= (dE t1) 1) 
(= (di t1) 2) 
(s t1) 
(= (r t1) 3) 
(park t1) 
(= (dtl) 3) 
(typep t1 'empty) 
))) 


In this program we create and initialize a solo machine, t0. As always, the 
first level machines must have at least one value. Then we create a status 
machine, t1. We park the head of the status machine, and then delete the 
leftmost instance on the tape, i.e. the 1. 


According to our convention though a parked head is not on any cell, the right 
neighbor of the parked head is the leftmost cell of the tape, so the second 
delete also deletes the leftmost cell, which now holds a 2. 


In the next line we step the machine. Stepping the machine will move the 
parked head to its right neighbor, i.e. the leftmost cell of the tape. We read 
that value as a check, and find that it is 3. Then we park the head again, and 
then delete the last cell. That causes the machine to become empty. 


The initial status of a machine can be created set to empty using :status 
‘empty, or to parked by using :status 'parked in the initialization line. First 
level machines must have at least one cell, so when we want an empty status 
machine, we first create a base machine with a single cell holding an instance 
of Ø. We could have used any value. When a status machine becomes empty, 
including in the initializer, the instance for the last cell is set to @ to facilitate 
garbage collection of the last instance. Adding a cell to an empty machine 
causes the machine to transition to the parked status. 
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(defun test-status-6 () 
(let*( 
(tm10 (mk '‘list-solo-tm {:tape {0}})) 
(tm11 (mk 'status-tm {:base tm10 :status 'empty})) 
(tm20 (mk '‘list-solo-tm {:tape {0}})) 
(tm21 (mk 'status-tm {:base tm20 :status 'empty})) 


) 

(A 
(a tmii 101) 
(a tm11 102) 
(a tm11 103) 
(typep tm11 'parked) 
(di tm11 tm21) 
(di tm11 tm21) 
(di tm11 tm21) 
(= (r tm21) 101) 
(cl tm21) 
(= (r tm21) 103) 
(s tm21) 
(= (rtm21) 102) 
(s tm21) 
(= (rtm21) 101) 
(= (s tm21)) 
) 

)) 


Entanglement accounting - ea-tm 
ea-tm is a specialization of status-tm. 


When multiple machines share a tape, we say they are entangled. There is a 
synchronization problem when a set of machines are entangled. If any 
member of that set becomes empty, all members of the set must become 
empty. Typically machines keep track of their tape through a reference to the 
leftmost cell, so if any has a new leftmost cell, or the leftmost cell is 
deallocated, all must update their leftmost cell reference. Before deleting a 
cell from the tape while using one machine, we must check that no other 
machine has a head on that cell. ea-tm is a specialized status machine that 
adds an entanglement accounting system to keep track of all the above. 


Entanglement accounting makes it safe to use a tm-haz machine as a base 
machine. This is good news, as both the solo-tm and nd-tm interfaces 
become available simultaneously without hazards. 
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(def-type ea-tm (status-tm) 
( 


(entanglements 
tinitarg :entanglements 
:accessor entanglements 


(entanglements-pt ; our location in the entangelments list 
:initarg :entanglements-pt 
:accessor entanglements-pt 
) 

)) 


(def-type ea-parked-active (ea-tm)()) 


(def-type ea-abandoned (abandoned ea-tm)()) 

(def-type ea-active (ea-parked-active active ea-tm)()) 
(def-type ea-empty (empty ea-tm)()) 

(def-type ea-parked (ea-parked-active parked ea-tm)()) 


The entanglements slots (fields) are only used internally by the ea-tm 
machine. Each instance on the entanglements list is an ea-tm machine that 
shares the tape. entanglements is a sort of tape listener list. 
entanglements is a bilist-haz-tm machine and is shared by all tape users. 
entanglements-pt indicates the location in the blist-haz-tm for the given 
machine. We don't actually need both of these fields, because 
entanglements can be recovered from entanglements-pt. (We simply 
make a copy of it, and cue it to leftmost.) 


The abandon function changes a status machine's status to abandoned, 
and for an ea-tm machine it also removes it from the listener list. An 
entangled machine created using with-entanglements will be 
automatically abandoned when it goes out of scope. 


(abandon tm) 


(with-entangled tm-orig (A(tm-entangled) body*)) 


I tried weak pointers instead of having an explicit abandon function, but the 
weak pointers did not change to @ when an entangled machine went out of 
static scope, say after a let form, but rather became Ø only after they were 
garbage collected. I probably should have expected this, as most instances in 
Lisp have dynamic extent, they stick around until they are no longer used. 
We only know they are no longer used when the garbage collector runs. This 
caused machines to remain entangled in places the programmer did not 
expect them to be. In the case of status machines, one could park the 
machines when done with them, so this latent existence wouldn't cause a 
problem with collisions, but I had to ask, if I'm going through the effort to 
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park machines when I'm done with them, why not just abandon them? 


An additional continuation is added to the interface, that of collision. This 
is invoked if one entangled ea-tm machine attempts to delete the cell that 
another entangled ea-tm machine has its head on. It also makes a nice proof 
target, in some situations, if a person can prove that there can be no 
collisions, then one can use simpler machines. The delete function ends up 
looking like this: 


(defun-typed d ((tm ea-active) &optional spill >) 
(destructuring-bind 
(&key 
(70k #'echo) 
(>collision (A()(error 'dealloc-collision))) 
&allow-other-keys 


) 
=> 
(labels( 
(dec-rightside-addresses (tm) 
(civ* (entanglements tm) 
(A(es) 
(let( 
(etm (r es)) 
) 
(when etm 
(decf (address-rightmost etm)) 
(when (> (address etm) (address tm)) 
(decf (address etm)) 
)))))) 
(entangled-on-right-neighbor-cell tm 
{ 
:>t collision 


: >Ø (A() 
(d (base tm) spill 
{ 


:> ok (A(instance) 
(dec-rightside-addresses tm) 
[ok instance] 
) 
:>collision #'cant-happen 
(o (remove-key-pairs > {:-0k :-collision})) 
aD) 
+)))) 


First we call the predicate entangled-on-right-neighbor-cell to check if 
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any machine in the entanglement set has a head on the soon to be deleted 
right neighbor cell. If so, we take the collision continuation. If not then we 
know it is safe to delete the cell, even though the base machines might be of 
type tm-haz. In this implementation of status, we also maintain addresses, 
so after the cell is deleted, we decrement the addresses fields for all the 
entangled machines that have a head to the right of the cell that was just 
deleted. We can easily check which they are as we know the address the head 
is on for our own machine. 


The entangled-on-right-neighbor function is just a question of existence 
for a machine with an address one greater than ours. We are careful here not 
to create an intermediate address that can't be represented in the same word 
size as the intermediate bits. Though that is not necessary in Lisp because 
the addresses are bignums. Still there is nothing wrong with using good 
form. 


(defun-typed entangled-on-right-neighbor-cell ((tm  ea-active) 
&optional >) 
(destructuring-bind 
(&key 
(>t (be t)) 
(>Ø (be @)) 
&allow-other-keys 
) 
=> 
(cla (entanglements tm) 
(A(es ct cø) 
(let( 
(etm (r es)) 


etm 
(typep etm ‘active) 
(# (address etm) 0) 
(= (address tm) (- (address etm) 1)) 
) 
[ct] 
[cø] 
))) 
>t 
>Ø 
)) 


Using d to delete cells from an active machine can never make the machine 
empty. 
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This is because d deletes the right neighbor cell, so we can never delete 
leftmost. Delete leftmost, dil cannot make an active machine empty either, 
as the head will be on some cell, and trying to delete that cell will result ina 
collision. However, status machines can be parked, and a parked machine has 
no head on the tape. If any of the machines in the entanglement set are 
empty, then all the machines are, after all they share the same tape. It follows 
that to make a machine empty, it, and all of its entanglements, must be 
parked. 


Here is the code for di ona parked machine. This occurs after the definition 
of three helper functions, which I have omitted for brevity's sake. 


(entangled-on-leftmost (entanglements tm) 
collision 
(AQ) 
(let( 
(spill-instance (eciir (base tm))) 


(labels( 
(delete-0 () 
(if (= (address-rightmost tm) 0) 
(make-empty) 
(progn 
(step-parked-machines) 
(di (base tm) @ 
{:>0k (A(instance) 
(declare (ignore instance)) 
(fix-tapes-dec-addresses) 


) 
:>collision #'cant-happen 
:>no-alloc #'cant-happen 


¥))) 
[ok spill-instance] 


) 
) 
(if spill 
(as spill spill-instance 


:90k #'delete-0 
:>no-alloc >no-alloc 


}) 
(delete-0) 
) 
D) 
We start with an existence question similar to that for d, does there exist a 
machine in the entanglement set that has its head on leftmost? If not, we 
begin by recovering the instance that is to be spilled. When then look at the 


address of rightmost. If it is address zero, then we make all the entangled 
machines empty. Otherwise we delete the leftmost cell of the machines. We 


104/148 


run into a small problem though. The base machines are treated like level 
one machines, so for the parked machines we have left the base machine 
heads on the left most cells while ignoring them. To compensate for this, we 
scan though the entanglements set looking for such cases, and if we find one, 
we bump the head to the right. This will always be possible because at this 
point we know the machines will not be empty after the delete. 


Here is the code for make-empty. 


(make-empty () 
(w (base tm) @) 
(cliv* (entanglements tm) 
(A(es) 
(let( 
(etm (r es)) 


(when etm (to-empty etm)) 


)))) 


It is defined in the labels section, so there is no defun keyword. The first 
thing that it does is to write @ to the leftmost cell of the base machine. We do 
this so that the instance held in the last cell will be garbage collected. We 
don't actually delete the last cell because that would violate the constraints 
for level one machines, which must always have at least one cell. Keeping the 
last cell around also makes it easy to restart an empty machine by just 
writing a new instance to this last cell and changing the status to active. 


Once the last cell's instance is freed, we walk through the entanglement list 
and call to-empty on each machine. to-empty is a function class, and it will 
be matched with an implementation at dispatch time. For ea-tm machines it 
will call this one function: 


(defun-typed to-empty ((tm ea-tm)) (change-class tm 'ea- 
empty)) 


Both ea-active and ea-empty are inherited from ea-tm and neither have 
any additional slots, so we expect the change-class to be safe and not to 
require much work. All off the machine states (statuses) have analogous 
functions. These are primitive instructions that are used internally. There are 
higher level versions for abandoning and parking machines available to the 
programmer. Other status changes occur as side effects of operations. 


Machines often reference the tape via a pointer to the first cell in the tape. If 
we orphan the first cell from the tape list, the other machines will no longer 
be able to read the tape. It would be sufficient to call dH on each of the 
entangled machine, but instead we chose to add a function to the first level 
machine's interface to tell each machine that there was a delete, and to hand 
it a copy of the machine with the delete, (update-tape-after-dI (base 
etm) (base tm)). This is found inside of fix-tapes-dec-addresses. 
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Analogously when we have multiple machines sharing a tape, and we 
prepend a new cell to the front of the tape on one machine, the other 
machines would not know about this change, and thus would have a 
different, erroneous, view of the tape. This is why prepend is not acommand 
for tape-machine or nd-machine. Hence we have an analogous function, 
(update-tape-after-all (base etm) (base tm)) that is run after a cell is 
prepended. 


The existence of the update commands causes us to fall a little short of the 
goal whereby the second level machine uses the programmer interface of the 
first level machines without modification. Though, this may be indicative of 
the fact that we need to add some cell sharing functions to the first level. 
Currently the only function that explicitly specified to move cells is d with its 
spilling feature, though the cell moving part of that specification is optional, 
as it must be, because implementations such as arrays cannot transfer cells. 


Thread Safe - ts1-tm 
tsi-tm is a thread safe specialization of ea-tm. 


As wonderful as the ea-tm is, it still does not guarantee thread safety for 
status machines. By 'thread safety' we mean that the interface will continue 
to perform to specification for any order and timing of interface function 
calls. Though thread safety is a great attribute for the library, this feature 
does not help the programmer to synchronize his or her own data. The 
programmer must still work that out, rather here we just want to make sure 
that the library doesn't break. 


We can already have thread safety with the nd-tm. Though it is hard to 
imagine implementing such a thing as a production code pipe between 
processes non-destructively. Perhaps if we sent multiple machines as a 
bucket brigade, as after all, nd machines can be destroyed as whole units by 
deallocating them. 


Any of the machines may be used in a thread safe manner if we guarantee 
that only one thread at a time has control. Emulated threads are not good 
enough in this respect, as they time multiplex at too low of a level, so an 
interface function may not complete before the next thread runs. Rather we 
need something like a mutual exclusion lock on the machine, where the 
owner of the lock may use the machine, and the lock is only exchanged at the 
same level of execution as the function calls. 


When one thread is deleting a cell from a tape, other threads with entangled 
copies of the machine cannot move their heads over said cell because they 
might run off the tracks, so to speak. With arrays that have deleted cell 
markers it is dangerous to read a cell that is being deleted as the deleted cell 
will have its instance set to null. When two entangled machines on differing 
threads delete different cells, there can be problems if the cells are 
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neighbors. We have analogous problems on the entanglement list, and this 
reflects up through the hierarchy, so that it is not safe to simultaneously 
abandon two machines (because this corresponds to entanglements list 
deletions), nor to simultaneously abandon a machine and delete a cell 
(because the cell delete will read the entanglements list while the abandon 
may be deleting a cell). We also cannot allow head motion on any entangled 
machine during collision checks, or the collision check may turn out 
incorrect. The entanglements list is also traversed when prepending a new 
leftmost cell, and when setting the entangled machines to the empty state, or 
moving them from empty to parked. 


With the ts-tm two entangled machines may safely run simultaneously on 
separate threads. An example would be one process putting data into a pipe, 
while the other pulls it out. Perhaps they start stepping on each other's work 
in the middle. We don't want anything to break should that happen. 


The simplest algorithm for creating thread safety, which we call ‘algorithm 
1', is to acquire a common lock for each interface call. Thus each interface 
function would run exclusively of the others. This simple approach has low 
overhead. Overhead is important, because in general, tape machine interface 
functions don't do much. I explored a couple of more complex algorithms 
that provided for more parallelism, but it wasn't apparent that under typical 
use they would be faster. 


Hence, ts1-tm simply wraps each call in a 'with-recursive-lock-held' 
environment. For example: 


(defun-typed s ((tm tsi-tm) &optional >) 
(bt: with-recursive-lock-held ((deed tm)) 
(call-next-method tm >) 


)) 


We use recursive locks because some functions call other functions in the 
library. As examples, generic functions are composed entirely of calls to 
other functions. We would like to have called the less specialized version of 
these other functions that didn't have the lock wrappers, and thus avoid the 
redundant lock setting, but CLOS does not provide a facility for doing this. 


Streaming 


Here is a little worker program used for streaming computation, when called, 
it squares the number on tm-source, and puts the result on tm-sink. 


(defun square-worker (tm-source tm-sink) 
(as tm-sink (expt (r tm-source) 2)) 
(s tm-source) 


) 


Suppose we have an infinite input stream, and want to process, say, the first 
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five instances. Here we set the input stream to the Natural Numbers. 


(let*( 
(tm-source (mk-Natural)) 
(tm10 (mk ‘list-nd-tm {:tape {0}})) 
(tm-sink (mk 'status-tm {:base tm10 :empty t})) 
) 
(v* (mk-interval 0 4) 
(A(tm)(declare (ignore tm)) 
(square-worker tm-source tm-sink) 
)) 
(tm-print tm-sink) 
) 


We use universal quantification over an interval of five values to affect our 
count. Upon each count square-worker will do its job. For status-tm 
machines the function tm-print is currently defined to print the status 
followed by a pair separated by a colon, the first in the pair being the head 
address, the second being the address of rightmost. This is followed by the 
contents of the tape. Square brackets appear around the instance 
representation corresponding to the cell the head is on. This is ad hoc, and 
likely to change in a later version. In the current version our output appears 
as: 


ACTIVE (4:4)0149 [16] 
a 


Notice that we had to add (declare (ignore tm)) after the lambda on the 
line with the quantifier. We have no choice, as the quantifier always passes 
us a copy of the machine, but Lisp thinks there is something funny if a 
variable is passed into a function, and is not used, so we have to tell it we are 
ignoring the variable. We can turn this off, but in other contexts it is useful 
information. I've thought about getting the quantified tape machine from the 
closure, but we often use this feature for giving a name to an evaluated form. 


Also notice that square-worker is called with the same input arguments 
repeatedly. We can make this code cleaner by currying those variables out of 
the loop: 
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(let*( 
(tm-source (mk-Natural)) 
(tm10 (mk '‘list-nd-tm {:tape {@}})) 
(tm-sink (mk 'status-tm {:base tm10 :empty t})) 
(worker (A()(Ssquare-worker tm-source tm-sink))) 


(v* (mk-interval 0 4) 
(A(tm)(declare (ignore tm)) 
[worker] 
)) 
(tm-print tm-sink) 
) 


Now it is clear that no loop variables are being passed into worker. This is 
the preferred form. In the previous version of the library, we supported this 
form with macros, but it can be done equally well with good programming 
style, so the macros are unnecessary. 


We can think of worker as a circuit that has been wired in. tm-source is the 
input over time, tm-sink is the output, and the quantifier is providing the 
clock signal. This is the high level programming model I developed for my 
reconfigurable data path streaming processor at Quicksilver Technologies. In 
that case, each configuration command corresponds to a different worker 
function. 


Transforms 


A transform is a tape machine that, like the second level machines, has a base 
machine. Its job is to make that base machine behave a little differently. 


Identity 


The most boring transform is the identity-tr, which simply passes ever 
operation to the base machine. This machine has a fully specified interface, 
which makes it useful as a fall back other more specialized transforms that 
might not have a full interface. 


Affine 


The affine-tr transform is a specialization of the identity transform that 
makes the base machine tape appear to be circular. Affine implements 
functions that have a leftmost or rightmost continuation, or affect their left 
or right neighbors, and then hooks into those. So, for example, when the base 
machine is stepped from rightmost, the affine machine cues the base 
machine to leftmost and takes the ok continuation. 


As of version 0.7 the old affine code has been deprecated, and we haven't 
updated it yet. 
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Ensemble 


The ensemble transform makes a group of machines appear to be a single 
machine. The individuals to be grouped can be provided either as a list or as 
instances on another machine. In the example below we use the :list 
initializer. So, for example, stepping an ensemble will in turn step all the 
member machines. The prior version implemented transactional behavior, 
i.e. if any of the machines in the group would take rightmost, then none of 
them would step, and the continuation would be taken. In contrast, this 
version walks down the list of member machines, stepping each, and if any 
takes a rightmost continuation, then the ensemble takes the rightmost 
continuation. When the rightmost continuation is taken, the internal 
members machine will have its head on the cell holding the instance of the 
machine that took the rightmost continuation. Here is an example of using 
ensemble, this test, like all our tests, is intended to return t. 
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(defun test-ensemble-0 () 
(let*( 
(tmO (mk '‘list-tm {:tape {1 2 3 31}})) 
(tmi (mk '‘list-tm {:tape {4 5 6}})) 
(tm2 (mk '‘list-tm {:tape {7 8 9 91}})) 
(tm10 (mk 'ensemble {:list {tmO tm1 tm2}})) 
) 
(A 

(= (rtm) 1) 

(= (rtmi1) 4) 

(= (r tm2) 7) 

(s tm10) 

(= (rtm0) 2) 

(= (r tmi) 5) 

(= (r tm2) 8) 

(s tm10) 

(= (r tm0) 3) 

(= (r tmi) 6) 

(= (r tm2) 9) 

G (s tm10)) 

(eq (r (members tm10)) tm1) 

(on-rightmost (r (members tm10))) 


(co (members tm10)) 

(on-rightmost (r (members tm10))) 

(= (r (r (members tm10))) 31) ; this machine stepped 

(s (members tm10)) 

(on-rightmost (r (members tm10))) 

(= (r (r (members tm10))) 6) ; this machine failed to step 
(s (members tm10)) 

(= (on-rightmost (r (members tm10)))) 

(= (r (r (members tm10))) 9) ; haven't stepped this one yet 
(= (s (members tm10))) 


(ca (members tm10) 
(A(tm ct cØ) 
(if 

(= (on-rightmost (r tm))) 
[ct] 
[cø] 
)) 

)) 
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TM Access Language 


The TM Library consists of many functions with single letter names, or a 
single letter and a symbol. This is so they can be easily composed to make 
longer commands. It is already the case that when we had a function that 
performed the same thing that could be done from multiple more primitive 
functions, we gave ita compound name, for example, esr. However, functions 
like esr potentially have their own implementation, the compounding is 
simply used to provide a descriptive name. Here with the access language 
stringing the letters together is writing a little program that instructions Lisp 
to run each command one after the other. 


(6 sn-snr tm 3 4) 
(A sn-snr tm 3 4) 


0 is the function (interpreted) form, and A is the macro (compiled) form. We 
read from left to right, and parameters are removed from the Iparameters 
list at the right in order as needed. Continuation options are added as 
appropriate. No continuations are shown in the example just above. This 
access program says to step the machine right 3 places, and then to step it 
left four places, and then to read the instance. This program is not identical 
to just stepping left by one, because right stepping might hit a right bound. 


As of version 0.7 we have not implemented the TM Access Language 
interpreter or compiler. 
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TM Library summary 


Types 
list-tm 
list-solo-tm 
list-nd-tm 
list-haz-tm 


bilist-tm 
bilist-solo-tm 
bilist-nd-tm 
bilist-haz-tm 


recursive 


identity-tr 
ensemble 


status-tm 
ea-tm 
ts1-tm 


Interface 


tm 


(defun mk (tm-class &optional init-parms >) 
(def-function-class eciir (tm &optional —>)) 
(def-function-class ecisr (tm &optional >)) 
(def-function-class eclisw (tm instance &optional —)) 
(def-function-class ecw (tm instance &optional >)) 
(def-function-class esr (tm &optional >)) 
(def-function-class esw (tm instance &optional >)) 
(def-function-class init (instance &optional init-value —)) 
(def-function-class r (tm &optional >)) 
(def-function-class w (tm instance &optional —>)) 


tm-generic 


(def-function-class a&hii (tm instance &optional >) 
(def-function-class as (tm instance &optional >) 
(def-function-class as&hii (tm instance &optional >) 
(def-function-class că (tm &optional =>) 


solo-tm 
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(def-function-class all (tm instance &optional >) 
(def-function-class d (tm &optional spill =>) 
(def-function-class di (tm &optional spill >) 
(def-function-class update-tape-after-all (tm tm-ref)) 
(def-function-class update-tape-after-di (tm tm-ref)) 


solo-tm-generic 
currently there are none. 
nd-tm 


(def-function-class ali (tm instance &optional >) 
(def-function-class entangle (tm-orig &optional —>)) 
(def-function-class entangled (tm0O tm1 &optional —)) 
(def-function-class heads-on-same-cell (tm0O tm1 &optional —)) 
(def-function-class s+ (tm0 tm1 &optional >) 
(def-function-class with-entangled (tm continuation)) 


nd-tm-generic 


(def-function-class tm-print (tm)) 


haz 


(def-function-class d. (tm &optional spill >) 


bi-tm 


(def-function-class -s (tm &optional >) 
(def-function-class -a (tm instance &optional =>) 
(def-function-class -d (tm &optional spill >) 


quantifiers 


(defun always-false (tm >t >) 

(defun always-true (tm >t >) 

(defun civ (tm pred &optional (>t (be t)) (>Ø (be @))) 
(defun clv* (tm function) 

(defun cid (tm pred &optional (>t (be t)) (>Ø (be @))) 
(defun ci4a* (tm pred) 

(defun v (tm pred &optional (>t (be t)) (>Ø (be @))) 
(defun v* (tm function) 

(defun 3 (tm pred &optional (>t (be t)) (>Ø (be @))) 
(defun 3* (tm pred) 

(defun © (work) 
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quantified 


(def-function-class -s* (tm) 
(def-function-class a* (tm fill &optional >) 
(def-function-class as* (tm fill &optional >) 
(def-function-class asn (tm n &optional fill >) 
(def-function-class s* (tm) 

(def-function-class sn (tm n &optional >) 
(def-function-class w* (tm fill &optional >) 
(defun-typed -s*((tm tape-machine))(cl tm)) 
(defun-typed a* 

(defun-typed as* 

(defun-typed asn 

(defun-typed s* ((tm tape-machine)) (ci tm)) 
(defun-typed sn 

(defun-typed w* 


recursive 


(defun increment-to (b &optional (stride 1)) 
(defun decrement-to (a &optional (stride 1)) 
(defun mk-interval (a b &optional (stride 1)) 
(defun mk-Natural () 


status 


(def-function-class abandon (tm)) 
(def-function-class park (tm &optional —)) 
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Review of the C memory model 


This chapter is a review of the C memory model and the nomenclature used 
for talking about it. 


If you have read this book up until this point you have probably noticed that 
an important component of the TTCA is its memory model. The TTCA 
memory model differs from the C memory model in some subtle, and some 
not-so-subtle, ways. If we are to discuss these differences, we will first need 
to have a firm grasp of the C memory model. 


Persistence 


The adjective static applies to things resolved by the compiler, linker, or 
perhaps loader; or by an interpreter without taking into account run time 
state. 


The adjective dynamic means something is resolved by executing the 
program, or at least will potentially be resolved that way. 


In the C language lexical scope is a static concept. We can see it when we read 
our program source, but it will be gone by run time. At run time we will 
instead have dynamic scope for such things as stack frames and variables. 


The term initialization is a synonym for the first write after data has been 
allocated. When data is exposed after allocation and before the first write, 
there is a hazard that uninitialized will be read, i.e. a read before initialization 
hazard. Generally, code is malformed if it reads data before initializing it. The 
term ‘static initialization’ means that data was given a value at compile time. 


The C language provides syntax for the initialization of data, though what can 
be done with these C ‘initializers’ is quite limited. When an initializer is 
applied at compile time to statically allocated data, we say that the data has 
been statically initialized. 


If data is never written again after being initialized, we say that the data is 
‘constant’. Using statically initialized constants (i.e. constants created at 
compile time) often provides the compiler with an opportunity for 
optimizing code. 


The offsets for the fields of a struct are statically initialized constants, and 
may be accessed through the offsetof macro. The compiler may place these 
values directly into the program text, i.e. into the instructions that constitute 
the program. 


If we have a ‘mutability’ service, we might be able to set data to be read-only 
for some duration of time, and not read-only during other times. It is 
common for people to refer to read-only data as being constant; however, 
when mutability can be changed, the compiler typically cannot take 
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advantage of read-only status, so it is not really useful to say that read-only 
data of this type is ‘constant’. 


The state machine controller of a Turing Machine is static and constant. This 
is noteworthy because our program is the controller definition for our 
processor. Hence, we would not expect that requiring a program to be a 
statically initialized constant would limit its computational power. 


Allocation and data 


A memory allocation is a block of memory set aside by a memory manager 
such as malloc, which may safely be used by the program. The memory 
manager maintains a contract with its users that it will never allocate the 
same memory to any two such active allocations. A memory allocation 
becomes inactive after being a call to deallocate it, which in C is done with 
the function free. 


An allocated block of memory is analogous to a physical memory device. Like 
for physical memory, an allocation has a fixed size, and data may be written 
to it and then later read back. On some computing systems physical memory 
may be plugged in and removed, just as amemory manager may be called to 
allocate and deallocate memory. 


Data refers to the values of the bits found on a memory device, and by 
extension, those found within a memory allocation. 


Data may be moved using a copy operation. A copy operation sets the data at 
some destination allocation so that it matches the data at a source allocation. 
The data at the source is not affected, while the data formerly at the 
destination will be clobbered. Because the source data remains intact, it is 
possible to copy source data to multiple destinations. An exception to this is 
the case where the source and destination are the same. The behavior in such 
a case is generally undefined, but might be defined for specific systems. 


Read and write are not distinct operations from copy, but rather these terms 
describe what happens on one end or the other of a copy operation. 


The most common types of allocation are: static, program stack base, 
program stack, and heap. 


All allocators have two main operations on their interface, allocate and 
deallocate. We say that an allocation is alive after it has been allocated and 
before it has been deallocated. 


The allocation and deallocation operators are commonly shortened to alloc, 
and dealloc. It is also common that allocators of different types use different 
names for the allocation and deallocation operators. 
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Register 


Most all processors have a small internal memory called a register file. Each 
register is typically given a name, and in the compiled code this name 
reduces to an integer, i.e. to an address. The general use registers are 
typically allocated and deallocated by the compiler. Some special purpose 
registers, such as a stack pointer, will typically not participate in the 
allocation scheme. This happens invisibly to the programmer. 


However, even the assignments made by the compiler might not be where 
data is actually put. It is common on modern processors to attempt to 
reconstruct a more parallel execution friendly data flow graph on the fly. 
False dependencies are removed by using what amounts to a larger register 
file and then renaming registers, i.e. mapping them different addresses in the 
larger register file. Ifthe processor is friendly, it will retire results in the same 
order as they would have been retired in the original program without the 
renaming, so that visible state remains the same. 


Hence, programmers really do not have much control over register 
allocation. 


Static 


The compiler will create a symbol table that associates identifiers with the 
locations of data in memory. Each location will be either an absolute address 
or a relative offset, but in either case it will be a constant. In the C language 
paradigm, the goal of the compiler, linker, and loader, is to embed these 
constant location values into the instructions of the program. Unless we have 
told the compiler to keep symbols as debugging information, they will be 
gone at run time. 


Data that has an absolute address determined at compile time is said to have 
been statically allocated. Such data is only deallocated when the process 
terminates and releases its virtual memory space. For statically allocated 
data all threads will see the same data located in the same place. 


Program stack 


The program stack is an instance of a stack data structure that is 
automatically managed though code inserted by the compiler and actions 
taken by the operating system. Each thread of execution is given its own 
program stack, which is a copy of the stack that the thread was forked from 
when it was created. Conventionally this stack copy is done in an efficient 
manner by using the virtual memory system and copy on write pages. 


When the program stack is used for memory allocation the allocation 
operator is called push and the deallocation operator is pop, occasionally 
someone will instead call these operations push and pull. For the stack used 
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by C, data is pushed in blocks known as a stack frames. Each stack frame will 
hold the arguments and local variables to be used by the function plus some 
overhead information such as the return address to continue from after the 
call. 


Static data and program stack base data live for the duration of the program. 
In contrast, stack allocated data comes to life at run time when a function is 
called and the corresponding stack frame is pushed, and its life ends when 
the function returns and the corresponding stack frame is popped. (This 
concept of lifetimes becomes important when we consider memory usage, 
including register usage by programs when compiling. We find that there are 
both static and dynamic concepts of allocation lifetimes.) 


In C a lexical scope starts with an opening brace, and ends with a matching 
closing brace. At least in some compilers, although scope may be nested 
within a function, all declared local variables will be placed in the stack frame 
of the containing function. Thus, stack frames are specific to functions, and 
not specific to lexical scope. The enforcement of the nesting of lexical scope 
within the function is then due to the compiler itself refusing to allow a 
programmer to make use of out of scope variables. Thus there is a difference 
in C between static scoping and dynamic scoping of data. This approach is 
one method for assuring the clean deallocation of local variables even when 
such things as a goto can cause a program to leave a scope level. It also 
causes allocation to be done only once even though a braced lexical scope 
with variable declarations occurs inside of a loop. 


The compiled code will address arguments and local variables at constant 
offsets relative to the base of the function's stack frame. Functions use stack 
relative addressing to find their variables; hence the same code will be 
usable without modification against different stack frame instances. Le. 
arguments and local variables are accessed indirectly through the stack 
frame pointer. 


However, when we use the ampersand operator, ‘&’, on a local variable we 
will get its absolute address. No matter how far down in the call nesting we 
go, this absolute address will still locate the same value. Placing such an 
address in a generally accessible data structure creates the hazard that the 
address might be locating data that has been deallocated (due to the relevant 
stack frame having been popped), or even that it references allocated data 
from a new allocation (a new unrelated stack frame has been pushed). 


Most programs are written by composing a large number of small functions. 
Functions need to have their arguments in the same place relative to the 
stack frame each time they are called. Consequently a great deal of effort will 
be spent moving data into stack frames before function calls. This problem 
inspired AMD to provide a ‘fast call’ mode where arguments are passed 
through registers. These moves must be correctly timed before function 
calls. Generally all this causes the stack to be a bottleneck for both 
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optimization and hardware performance enhancements. 


Program stack base 


With ‘program stack base’ allocation, our data is placed in the first stack 
frame. This is the one that will be allocated to the ‘main()* function in C. The 
first stack frame is special because it will never be popped off the stack while 
the program is running. This fact causes program stack base data to resemble 
static allocation because that also remains for the duration of program 
execution. However, there is an important difference between these two 
methods of allocation in that stack base allocated data is copied when a new 
thread is forked, whereas static allocated data is not. 


The astute reader will note that ‘program stack base’ and ‘program stack’ 
allocation schemes are a first-rest pattern applied to a single program stack. 


Heap 


With ‘heap’ allocation we have a set of library functions for allocating blocks 
of memory from a free list, then later deallocating them by putting them back 
on the free list. In C it is conventional to use the stdlib functions which define 
only one heap to be shared by all functions. Heap overhead such as the free 
list base pointer will be statically allocated. The one stdlib heap will be 
shared by all threads. Modern versions of the heap have mutex locks already 
built in so as to assure thread safety. 


Like static data, heap data is not copies when a new thread is forked. Heap 
allocation in C is accomplished by calling malloc, which then returns a 
pointer to the allocation. Programmers must be careful and remember that 
when copying such pointers, the heap data is not copied. Heap data can only 
be accessed from one place at a time within a single thread. However, in a 
multithreaded environment, if a heap pointer copy is given to multiple 
threads, it is conceivable that more than one threads will access the same 
heap memory simultaneously. This gets back to our discussion on shared 
tapes between multiple machines. 


Garbage collection (not part of the C memory model) 


Garbage collection is not part of the C memory model. Nor is it part of the 
TTCA memory model. However the technique has become so common that a 
treatise on memory allocation would be remiss for not at least mentioning 
it. 

With garbage collection, allocation of memory is done explicitly as for heap 
allocation. However, unlike for the heap management of memory, with 
garbage collection the deallocation of data occurs implicitly in the 
background. One method of accomplishing this is for the overhead of a heap 
block to include a reference counter. When no references are left pointing to 
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the allocation, the allocation is then put on a 'ready to be cleaned up' list. 
Periodically the garbage collector routine runs, typically on another thread 
in the background, and goes through the ‘ready to be cleaned up' list and 
deallocates all blocks listed. 


The advantage of this approach over the heap allocation approach is that 
there cannot be bugs of the 'woops forgot to deallocate it’ nature. Such 
‘memory leaks' are all too common in code that makes use of heap memory 
management. 


The C language and its standard libraries have no special support for garbage 
collection. Making use of a custom library for this would be tedious because 
the allocator would have to be told each time a pointer went out of scope, for 
example upon reaching a closing brace after being temporarily allocated. The 
calls would occur in the same places that, if we were using a heap, the 
deallocation calls would occur. Hence garbage collection with C would be no 
less error prone than heap allocation. 


The proposal to modify the C compiler to by adding calls upon hitting the 
close brace of a pointers scope is not sufficient because copies of pointers 
can be made, and in C, they can be made in obviousness ways. If a person 
works a little harder it is possible to limit syntax, find copies, and to track 
scopes so that it all works. However after doing all of that, the language is no 
longer C. 


I mentioned earlier that conventional processors are typically C machines, 
and you would be correct to surmise that they do not have hardware support 
for garbage collection. In contrast, most processors have hardware support 
for manipulating the program stack. 


Storage Allocation Class Keywords in C 


The C programming language specification speaks of ‘storage allocation 
classes’ rather than allocation types. The language has the storage allocation 
class keywords: register, static, and auto. In the early version of C these 
keywords corresponded to their namesake allocation types, with auto being 
program stack allocation. However over time register has become a hint, 
static has been overloaded with new meaning, and auto has become 
meaningless as in contexts where it may be used it is already the default. 


Specifying the register storage allocation class is a request to the compiler 
to place the given variable in a hardware register. It is rare to see the register 
storage class keyword used because it is almost always better to let the 
compiler decide what should go into a register, and besides that, the 
compiler is free to ignore the request. Perhaps this becomes important for 
special purpose compilers for embedded systems. 


Historically specifying a variable allocation with the static keyword would 
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cause the variable to be allocated statically in memory. This was and still is 
the default for variables declared at global scope even without the static 
keyword. The names of such variables will be listed in the object code, so 
they can be the targets of variables declared with the extern keyword. In old 
versions of the C language this used to be the end of the story. However today 
the static keyword found on variables at global scope now means that said 
variable is not visible outside the source code file it is found in. It is opposite 
of the old meaning as trying to link to it from another object file compiled 
with an extern declaration will not work. 


Functions in standard C are always declared at global scope. Declaring a 
function is identical to declaring a function pointer, and a function pointer is 
data. Hence the static keyword works the same on functions as it does on 
other data. 


If an inline function is declared static, and the compiler decides to ignore the 
request to inline the function, then the inline function instead becomes a 
regular static function. Such functions are declared in header files, so to say 
it is static is to mean it is a regular function linkable within the scope of the 
file the include occurs in. 


In C auto class variables have no external linkage. This typically means they 
are allocated on the program stack. This is the default for variables declared 
inside functions, including inside the main() function. It is rare to see the 
auto storage class keyword used because this is already the default behavior 
inside functions, and at global scope it is illegal to use it. 


Values and References 


In C a pointer is an address bound to the type for the data being addressed. 
There is no address type in C, nor any pointer keyword. Instead when 
declaring a pointer C has us make a type declaration with an asterisk 
proceeding the identifier.23 


di; int i; 
2. int k; 
B: int *j = &k; 


On line 1 of this example data is allocated with the size of an int. C leaves the 
size of an int to be determined by the compiler realization. Had we wanted 


3The actual case is more complex. Because C allows the programmer to force casting 
between types, programmers can pretend that a pointer is an address by casting it 
before each use. Historically when C programmers wanted to play this game, they 
would use a pointer to Char. More recently C added the void keyword. A pointer 
is an address bound to a type, so it follows that a pointer to a void type is the same 
as an address. The compiler then requires that pointers to Void type be caste to a 
non-VOid type before being used. 
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to specify the size for the int we could have instead included the appropriate 
header and used a type such as int32_t instead of int. 


If these declarations are placed at global scope the data will be allocated 
statically. If they are in the main function this data will be in the base frame 
of the stack, and if they are made inside another function, they will be in that 
function’s stack frame. 


The allocation on line 1 will be associated with the symbol i which will be 
viewable within the lexical scope of the declaration, whatever that happens 
to be. It is not shown here. On the second line we again allocate an int. 


Things get a bit more interesting on line 3 where we have another allocation, 
though this time we are allocating an address rather than an int. On my 
machine ints are 4 bytes and addresses are 8, so this line would create a 
larger allocation than each of the prior two. The compiler will take note that 
at this address in memory we find int type data. 


Line 3 also has initialization code. We initialize j to be the address of k. k is 
indeed an integer, so the type contract check will pass without an error and 
the compiler will generate the code to perform this initialization. 


However, all of this is a bit misleading, because at run time when the 
processor needs to load data from memory, it must know the address of that 
data. Take for example, in a load/store architecture a processor loads data 
by executing a load where the operand of the instruction is the address of the 
data we desire to load. Hence, all data in memory will be accessed through 
addresses, even when no pointer has been declared. When we allocate an 
address, we are often just making the address visible to our program rather 
than creating something new. Once the address is visible it may be passed as 
a parameter to a function etc. 


After data is loaded it will be in a processor register. If the data is part of the 
observable state of the program, and it is modified by the computation, it 
must be written back to memory. A store instruction performs the inverse 
operation to the load instruction. 


It is possible that a variable will not appear in memory at all and thus not 
have an address. As an example, this can happen when the compiler decides 
that the variable may be allocated in a processor register. As another 
example, it can happen when the optimizer eliminates i altogether. Thus if 
we take the address of i in our source code, we might be doing something 
that was going to be done anyway, or we might be forcing the compiler to not 
use a register for i, or even forcing it to not eliminate i. 


Given all of this, we can see that the C model for data and pointers is only an 
abstraction of what happens at run time. It is not necessarily inefficient to 
read a value by dereferencing a pointer rather than using it directly, as the 
processor might have to use an address for the load either way. We can 
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examine the assembly output of the compiler to get a better idea of how our 
variables were really handled, but even that might not be the full picture, 
because modern processors will prefetch values, make use of caches, and 
rename registers so as to increase performance. 
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Emulating TTCA native code using C 


In the prior chapter we used the Common Lisp language because of its ability 
to embody formal descriptions. We then used that formal model to explore 
properties of software. In this chapter we take a step towards implementing 
an emulator for the TTCA native programming language. Thus, this chapter 
has a fundamentally different objective. 


As of the time of this writing conventional processors are pretty much C 
language machines. Consequently, a good place to start when implementing 
our emulator is to develop a set of macros and techniques for expressing 
TTCA native coding styles in C. These macros and techniques will then 
morph into a library used by any future emulators, interpreters, or 
compilers. 


The difficulties we stumble across while developing these macros will serve 
to highlight the differences between conventional processor architectures 
and TTCA. Similarly the manner of speaking we develop for talking about the 
emulator code will help us better discuss the TTCA. And who knows, perhaps 
this code will run especially well on conventional processors also. It does 
look to provide some opportunities for deep optimization. 


Using a stack full of function specific frames is a horribly inefficient way to 
encapsulate run time state. Take for example coding the state of a loop in C 
versus that of a recursive call in LISP. State in C programming style is typically 
just a loop control variable, while in conventional Lisp programming style 
there will be a stack frame pushed corresponding to each iteration, and these 
stack frames will typically include function call overhead, local variables and 
other things that were needed to lead up to the recursive call, but not be 
needed after it. Yet, we push all that stuff, repeatedly. 


Furthermore, the existence of a stack causes optimization bottle necks, even 
for C programs. This affects hardware acceleration, compiler optimization, 
and execution models with multiple threads. The conventional compiler 
cannot see beyond the current compilation unit, i.e. source file, so the stack 
simply has to be supported. The actual needed context is irrelevant because 
the stack cannot be thrown away. A new thread gets a whole copy whether it 
needs it or not. 


The inline function feature has helped with compiler optimization. Inline 
functions are essentially text macros, and are included from header files, 
thus they are available to the compiler to analyze. However, this also leads to 
code duplication. 


Prior chapters showed examples multiple return path computation using 
continuation arguments. Those examples were in Lisp. In those examples, 
when a function executed and completed its work, it would then execute one 
of the continuations that had been passed in as an argument. 
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Although the implication of those examples was that we had an efficient 
forward moving computation, the actual case was that each of these calls 
pushed another stack frame and we would eventually have to pop them all 
off. This is OK for those examples because the point was to teach the method, 
not to provide a means for implementation. In this chapter we will fix this 
defect, and show actual stack free feed forward computation, albeit, written 
in C. 


Stack frame push and pop on function calls and returns are intrinsic to the C 
language, so if we do not want to muck with the compiler itself, we will have 
to call our continued functions using gotos. 


We will give our new unit of execution a descriptive name, an instruction 
sequence. We will have two types of instruction sequences, specific use and 
general use. 


Specific use instruction sequences make use of data within lexical scope, 
which in C includes the goto target labels. This type of instruction sequence 
is natural for a programmer to write when sitting in front of a terminal 
making something for his or her own use. 


General use instruction sequences are reusable. They are the sort of the we 
would find in a library. 


The code for this chapter may be found at: 
http://www.github.com/Thomas-Walker-Lynch/TM2x 


In the code examples, global variables are all caps with between word 
underscores. Type names and name space names are initial character 
capitalized in each word with words run together. Other identifiers are lower 
case with words separated by underscores. C does not have built in 
namespace syntax, so we will use a center dot character to separate 
namespace names from the more specific part of identifiers. 


Specific Use Instruction Sequences 


An instruction sequence starts with an SQ-def macro call. It takes the 
sequence name as an argument. It ends with a matching SQ-end where the 
name of the sequence must again be provided as an argument. We call an 
instruction sequence by using the SQ-continue(name) macro. Hence, an 
instruction sequence will have this form: 
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SQ-def(name)<{ 
<place code here> 


} SQ-end(name); 


The def macro gives the instruction sequence a goto label. An instruction 
sequence must be reached using a continue call. We should never 
sequentially flow into it. I considered calling abort() in the def macro just 
above the label, but instead elected to add a goto that jumps to the end 
macro. Hence a sequence may be defined anywhere, and it will effectively be 
ignored during sequential execution. 


Instruction sequences do not return. All instruction sequences must execute 
another SQ-continue statement or terminate the thread before reaching 
the corresponding SQ-end. 


The specific use instruction sequences defined in this manner have no 
explicit arguments passed in, but they may use any variable that is visible in 
the outer lexical scope. 


Each instruction sequence name must be declared as type SQ:-Sequence, 
twice, once just as name, and once as SQ:name. This is of course a hack. 
Due to compiler limitations, instruction sequences must be declared within 
a function. We will put ours inside of main. 


Implementation Notes 


Our instruction sequences are similar in some aspects to gcc’s nested 
functions; however they do not make use of gcc nested functions. Our macros 
do make use of gcc extensions related to labels, so gcc must be used with this 
code. 


Due to some C language restrictions on goto and labels, our instruction 
sequences may not be defined at global scope. This is probably due to C 
trying to make goto play well with stack based function calls, and the 
compiler wanting to see both the goto and the label in a single compilation 
unit. Hence for now our sequences must be encapsulated in a function. We 
will put them in main. Hence everything is going to be surrounded by a big 
main function definition. All this should make the optimizer very happy. 


We may call C functions from within a sequence. Currently this is a practical 
necessity because we do have instruction sequence libraries for things such 
as i/o. 


It is possible to pass instruction sequence names into functions by preceding 
the name with two ampersands, &&name. A function given such arguments 
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might then make use of a trampoline by returning one of the sequence 
pointers. Such a return value may be dereferenced with a *, and then 
continued from it. Actually I implemented the first version of these examples 
by doing this. It had the nice effect of that not everything had to go into main; 
however, I promised to show the reader code that does not use the stack 
during calls, rather than just looking like it doesn't, so I have removed this 
from the examples. 


Example 


The following is a simple example using two specific use instruction 
sequences. It is located in the examples directory of the git repository, and is 
called special_use_instruction_sequences.c. 
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#include <stdlib.h> 
#include <stdio.h> 
#include <stdint.h> 
#include <inttypes.h> 
#include "misc.h" 
#include "Sequence.h" 


int main(int argc ,char **argv){ 
SQ:Sequence reduce ,SQ-reduce ,expand ,SQ-expand; 


er eee a 


11. uint32_ti; 

12. if( argc != 2 || !sscanf(argv[1], "%" SCNu32, &i) ){ 

13. fprintf(stderr ,"expected argument that fits in uint32_t"); 
14. return 1; 


15.) } 

16. 

17. SQ-continue(reduce); 
18. 


19. SQ-def(reduce){ 
20.  printf("%" PRIUu32 "\n", i); 


21i. if(i == 1 ) return 0; 
22.  if( i & Ox1 ) SQ-continue(expand); 
23. (tl) eee ale 


24. SQ-:continue(reduce); 
25. }SQ-end(reduce); 


27. SQ:def(expand){ 

28.  uint64_t result = (uint64_t)i* 3 + 1; 
29. if( result > UINT32_MAX ){ 

SO: printf("exploded!"); 

31: return 2; 

32. 3 

33. i= result; 

34. SQ-continue(reduce); 

35. }SQ-end(expand); 


Figure 4: Specific use sequences for printing Collatz Conjecture numbers 


Here we have two instruction sequences, reduce and expand. The reduce 
sequence divides an even valued i by two, while the expand sequence 
multiples i by 3 and adds 1. Note on line 29 we declare a local variable called 
result. This variable will be allocated once in main’s stack frame, and only 
deallocated when main returns, i.e. when the program ends. Due to our use 
of program stack base allocation, had this code made use of multiple threads, 
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each thread would have its own version of result. However, we do not have 
multiple threads in this example. Here is an invocation: 


> ./specific_use_sequences 12 
12 

6 

3 

10 

5 

16 


1 


Notice on line 17 main continues into reduce. Because reduce is an 
instruction sequence it will never return to line 17. We will only move 
forward through the instruction sequence from this point. Processing 
proceeds forward as a propagating wave. 


i has the value 12, and on line 22 reduce discovers it is even, so it divides it 
by two using a right shift. Then on line 24 reduce continues into itself. Here 
we see something interesting. There is functionally no difference between a 
recursive call and a loop form. This is because nothing is placed on the stack 
in the recursive call, and we will never return from it. Itis not just that these 
two forms are functionally the same and one can be transformed to the other, 
rather itis that they are identical. 


On the second pass through reduce, i initially has the value six, so this will 
again be divided by two, and we will again re-enter reduce. On our third time 
through reduce, i will have the value 3, which is odd, so then we continue 
with expand. expand multiplies i by 3, adds 1, and then continues into 
reduce again. And thus, we continue in a forward manner until at some point 
on line 22 we discover that i is one. At that point we finish the program by 
returning from main. 


In this example, pumping data caused integers to get larger, but did not 
require that the data structures expand. Had we needed an expanding data 
structure we would have used the heap. 


Here all the only variable shared among the instruction sequences was i, and 
we allocated it on the program stack base. The program stack base allocation 
is thread safe. Each thread gets its own stack. Collectively we call the 
variables used by the instruction sequences a tableau. It is a sort of chalk 
board where we can write things then read them off later. We should put all 
shared variables inside of a struct and given this collection a name, 
something like tableau_0. Doing this would then lead to a discussion of 
managing tableaux, e.g. whether a tableau should be shared among threads, 
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etc. We will have such a discussion in the next section. 


General Use Instruction Sequences 


Specific use instruction sequences such as those shown in the prior section 
do not accept arguments. Instead they have hard coded explicit use variables 
found within one or more tableaux. In our prior section we had one implicit 
tableau which was identical to the lexical scope. 


Hard coding all the input variable locations has a serious limitation. Say for 
example, if we were to write a specific use instruction sequence that prints 
the value of a variable. Perhaps one that variously prints the values of, say, i, 
j, and k. Because our specific use instruction sequence would have to have 
the variable name hard coded into it, we would need three separate specific 
use instruction sequences, one for each of i, j, and k. This kind of thing is 
workable, it is roughly what a C inline function does; however in some 
situations code duplication leads to unnecessarily long programs. 


A conventional function call is usually some variation of the following. The 
processor first pushes the address of the instruction after the call on to the 
stack. A stack frame is allocated on the stack with space for arguments and 
local variables. Then the arguments are copied into it. The processor then 
jumps to the specified function entry point. The stack pointer is of central 
importance because the function itself uses it to find its arguments and 
temporary variable allocations. When the function finishes it pops the stack 
frame and pushes its result. The caller then pops the result, then pops the 
continuation address that it had pushed earlier, and then jumps to it. 


This conventional approach makes it so that each function has its own 
memory context, and its inputs are found in this context. However this comes 
at the price of having to copy input arguments in, being limited to one 
continuation after the function completes, and having one return value. The 
limitation of only one continuation and one return value causes 
programmers to generate a lot of ad hoc code for dealing with situations 
where multiple continuations are needed. Sometimes a programmer will be 
pressed for time and chose to not implement some of the continuation paths 
that seem unlikely. This leads to bugs when these cases happen to turn out 
to be needed. The throw and catch technique is an out of paradigm approach 
for dealing with this problem, and because it is out of paradigm there are 
complications in dealing with the cross stack frame jumps. Typically, 
programmers are told to use it only for unusual cases. 


Latent Linking 


A C program is first compiled one compilation unit at a time. Such a 
compilation unit is typically a source file along with its included header files. 
This results in an object file. Each object file will have embedded in it a 
symbol table that lists the unresolved symbols and where their values belong 
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in the code. For example, if a user calls an external function, the symbol table 
will have an entry that contains the function name and the location in the 
object files where the function address is to be written when it becomes 
known. The object code is binary, and we would expect such a place to be in 
the immediate field of a call instruction. 


There might be many compilation units, and thus many object files. Hence 
the second step is to call a program called a linker which reads the embedded 
symbol tables in the object files and matches them up and writes in the 
missing addresses into the code. 


Now suppose that for an instruction sequence we wait until the very last 
moment, just after the call to the instruction sequence, to do a final linking 
step. This combination of an instruction sequence with latent linking would 
make the instruction sequence generally usable. 


Going back to our print example, we would have one instruction sequence 
that prints an integer. However, the linkage of the integer to the instruction 
sequence would occur at time of call, and could be different upon each call. 
When called for printing the variable i, the blank address for the print 
variable will be filled with the address of i. When called for j, the address 
blank will be filled with the address of j. etc. 


The latent linking occurs so late that we no longer have symbols, but rather 
everything has been reduced to addresses. Hence the symbol table used for 
latent linking will be a list of address pairs, the first of the pair being a 
constant pointing at the allocation for the variable being linked in, and the 
second will be the location in the code where this constant is to be written. 
Each call has its own symbol table for performing the latent link step. 


The following is what a symbol table for a latent link would look like. 


1. typedef struct{ 

2. SQ:Ptr sequence; 

3. SQ-:Pairs *args; 

4. SQ-:Pairs *results; 

5. SQ-:Pairs *continuations 
6. } SQ:CallTable; 


Line 2 sequence is a pointer to the general use instruction sequence. Line 3 
is a list of pairs. The first element of each pair is the address on the tableau 
for an input argument. The second element of the pair is the location in the 
general purpose sequence where this address should be written. There are 
analogous pair lists for the results and continuations. 


It turns out that we only need one list, because all the fields here have the 
same form and use. The form is that of two constant addresses, where the 
first address is to be written at the location indicated by the second. 


When we say list here, we are not saying how that list is being implemented. 
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In conventional C code a person would expect to see a null terminated array 
of pairs. 


To make latent linking work on a conventional machine, we would need to 
copy a code template to a thread owned buffer so that we do not affect other 
thread's view of the code. Then we would perform the latent link by filling in 
the missing addresses for input arguments, results, and continuations. Then 
we would jump to the buffered code. Chances are we would have to change 
the permissions on the virtual page the buffered code is on before we would 
be allowed to execute the code. 


My goal here is to make it so that we can write some realistic TTCA code 
examples just based on the macros; so copying a block of code as data, 
manipulating it, then changing a data page to a text page would require 
special permissions, is more involved than I had hoped for. But never fear 
pointer indirection is here! In the next section we will show how to use 
pointer indirection so as to avoid modifying the instruction sequence code. 


Interpreted Symbol Table 


With the latent link method, the symbol table is used once so as to fix up the 
code immediately before we jump to it. Accordingly, only the code that does 
the call reads the symbol table. The instruction sequence itself has no 
reference to the symbol table. 


In contrast, with the interpreted symbol table method the instruction 
sequence refers to the symbol table at run time. Instead of having blanks that 
are filled in with addresses, the instruction sequence contains compiled in 
offsets into the symbol table, and these are dereferenced to find the 
necessary data. 


The following is such an interpreted symbol table. We also call it a link. The 
link terminology comes about from a data flow view of our program, where 
the instruction sequences are nodes, and these symbol tables describe the 
connections between the nodes. 


. typedef struct{ 
SQ:Ptr sequence; 
SQ:Args *args; 
SQ:Ress *ress; 
SQ:Lnks *Inks; 

. } SQ-Lnk; 


Here, line 3 args is a pointer to a block of pointers to the inputs, line 4 ress 
is a pointer to a block of results, and line 5 Inks is a pointer to a block of 
continuation links. Actually we could have just put all of these pointers into 
one big null terminated array. However, for purposes of illustration it is nice 
to see the structure and to refer to the fields in the structures instead of 
indexing into an array. 


ae ai ce al 
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For purposes of illustration in this book I should have added another field 
that points to a tableau allocation for temporary variables. Here temporary 
variables are the same as result variables that no downstream instruction 
sequence references. 


Here our latent linker program is called SQ:-continue_indirect. We pass in 
our interpreted symbol table. Equivalently, we can use a data flow 
perspective and say that we are following a link. 


SQ:continue_indirect(Ink) 


An instruction sequence will need to reference the link to find its inputs, 
outputs, and continuations. Only one instruction sequence may run ata time 
in a given thread, so for these examples I have opted to keep a thread local 
variable that points to the currently active interpreted symbol table, i.e. to 
the currently active link. 


SQ:Lnk *SQ:Ink; 


In the example GitHub code this declaration is found in the file 
Sequence:Text.h and you will see this file included toward the top of the 
main function so as to avoid any forward referencing problems. Note that 
the type name is capitalized, while the instance name is lower case. 


The continue_indirect macro sets the active link pointer before calling the 
target instruction sequence. The address of the active pointer is compiled 
into each general use instruction sequence. 


#define SQ-continue_indirect(Ink) \ 
SQ:Ink = (SQ-Lnk *)&(Ink); \ 
SQ-continue(*SQ:Ink->sequence); 


If our processor has speculative execution, the reuse of a thread local 
variable will be a bottle neck that the processor runs into when it speculates 
past instruction sequence call boundaries. This looks a lot like the 
conventional bottleneck when modifying the stack pointer; however there is 
an important difference. There is no stack behind the link pointer. It can be 
safely renamed just like any other variable held in a register. 


There are many ways to make a link pointer available to general use 
sequences. Among these alternatives, each general use instruction sequence 
can be given its own allocation for a link pointer. 
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Example 


The following code is for a general use special instruction sequence that does 
a kind of multiply that is useful when dealing with inclusive bounds. For 
example, it may be used to find the byte offset to the last element in an array. 
More specifically consider an array of int32_t type cells with a maximum 
index of 7, then 8 * 4 - 4 = 28 is the byte index of the first byte of the last 
element. Alternatively, this same index can be computed without risk of 
overflow as 7 * 3 + 7 = 28. The inputs shown here are a_0O, the extent of an 
array; and a_1, the extent of an element. The result is r. 


. SQ:def(Inclusive:mul_ext){ 


1 

2. 

3. Inclusive:30pLL:Lnk *Ink = (Inclusive:30pLL:Lnk *)SQ:Ink; 

4 uintl28_t t = *Ink->args->a_O * *Ink->args->a_1 + 
*Ink->args->a_0; 


5 if( t > address _t_n ){ 

6. SQ:continue_indirect(Ink->Inks->gt_address_t_n); 
Be op 

8. *Ink->ress->r = t; 

9. SQ:continue_indirect(Ink->Inks->nominal); 

10. 

ital } SQ-end(Inclusive:-mul_ei_bi); 

1 


On line 3 we cast the global SQ:Ink to the more specific type 
Inclusive-30pLL-Lnk. All of our example general use sequences start with 
such a cast. Here are the type definitions: 
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typedef struct{ 
SQ:Ptr sequence; 
Inclusive: 30pLL-Args *args; 
Inclusive: 30pLL-Ress *ress; 
Inclusive: 30pLL:Lnks *Inks; 
}Inclusive:30pLL:Lnk; 


typedef struct { 
address_t *a_0; 
address_t *a_1; 

} Inclusive: 30pLL-Args; 

typedef struct { 
address_t *r; 

} Inclusive: 30pLL:Ress; 

typedef struct { 
SQ-:Lnk nominal; 
SQ:Lnk gt_address_t_n; 

} Inclusive: 30pLL:Lnks; 

// retypes SQ-Lnk 


The Args and Ress structs will hold pointers into the tableau. The Lnks 
struct will hold links to continuation instruction sequences. 


The Inclusive-mul_ext sequence then accesses its arguments and results 
indirectly through the Ink. Hence, every link may specify different places for 
arguments to be gathered from, and different places for results to be written 
to. Note that Inclusive-mul_ext has two continuations, one for when all 
goes well, and one in case the multiply overflows. We call the all-goes-well 
continuation the nominal continuation. We call the overflow continuation 
gt_address_t_n which stands for ‘greater than the extent of an address 
type’. Note that our multiply only writes a result if the nominal continuation 
is to be taken. 


Here is an example using the general instruction sequence 
Inclusive-mul_ ext: 
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#include <stdio.h> 

#include "misc.h" 

#include "Sequence.h" 

#include "Inclusive.h" 

#include "Inclusive:-DataTypes.h" 


int main(){ 
SQ:Sequence 

nominal ,SQ-:nominal ,gt_address_t_n ,SQ-gt_address_t_n; 
#include "Sequence:Text.h" 
#include "Inclusive: Text.h" 


// result type Tableau 
address ta_0O = 541; 
address t a_1 = 727; 
address _tr; 


// make a link 
SQ:make_Lnk(mul ,Inclusive:-30pLL ,&&Inclusive:-mul_ext) 
mul_args.a_O = &a_0; 
mul_args.a_1 = &a_1; 
mul_ress.r = &r; 
mul_Inks.nominal.sequence = &&nominal; 
mul_Inks.gt_address_t_n.sequence = &&gt_address_t_n; 


SQ:continue_indirect( mul_Ink ); 


SQ:def(nominal){ 
if( r == 394575 ){ 
printf("glorious\n"); 
return 0; 


printf("wrong answer\n"); 
return 1; 
}SQ-end(nominal); 


SQ-def(gt_address_t_n)<{ 
printf("product unexpectedly overflowed\n"); 
return 1; 

}SQ-end(gt_address_t_n); 


} 


The include file Sequence-Text.h only holds the global SQ-Ink. The include 
file Inclusive-Text.h holds the definition for mul_ext, as shown above. 
These are included in main so that their data will be stack base allocated. 
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The macro SQ:make_Lnk makes instances of the arguments, results, and 
links structures. After those are made, we assign the values to their fields. 


Tableau Organization 


Multiple instruction sequences sharing a tableau is analogous to an object 
oriented programming model object. In this analogy each instruction 
sequence corresponds to a method, and the tableau corresponds to the 
encapsulated data. Relative to our function call discussion a tableau was 
analogous to a stack frame. These two analogies both work because they are 
both examples of the more general concept of memory context. 


A Tableau is a finite object. Programmers can create recursive structures, and 
if they do this, and it leads to expanding state, then that state should be 
captured in a heap managed data structure, such as a list, tree, or queue. The 
header to that data structure will be a finite object that can be placed on a 
tableau. 


Result Centric 


All reads and writes are actually data copies. Similarly, each instruction 
sequence input value is another instruction sequence result value. Yet there 
is an asymmetry. Each instruction sequence reads each input from exactly 
one upstream result. In contrast, each result might need to be distributed to 
multiple downstream inputs. 


Due to this asymmetry a good heuristic for organizing a tableau might be to 
keep results from each given instruction sequence together as a block on the 
tableau, and then use lists of pointers to gather inputs from all over a tableau. 
We call this a result centric tableau. A given instruction sequence then needs 
one pointer per input to find its inputs, and one pointer to find its result 
block. The total number of pointers needed is determined only by the 
properties of the given instruction sequence. 


Our instructions sequences will need some initial inputs. These can be 
placed in a block in our result centric tableau as though they are results from 
some unseen instruction sequence. Alternatively, the input pointers might 
reach into other tableau to locate input data. 


There can be a hazard with the result centric approach. This hazard occurs 
when a given instruction sequence is called a second time through the same 
link, and clobbers a prior result, but another, yet to be called, instruction 
sequence still needs that prior result. The simplest fix is to use a different 
link for the second call, this different link would point to the same sequence, 
and to a different result block. Another possible fix is to use a FIFO to hold 
results, rather than a single register. 
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Input Centric 


The dual of the result centric approach is to have a block of inputs and result 
pointers. Most people find this more intuitive. In the least it is convenient 
when sitting in a debugger to be able to see all the inputs together just before 
stepping into a sequence. 


With an input centric tableau, each given instruction sequence needs one 
pointer to find its input block, and a list of pointers for each result, so as to 
know where to distribute the result. Here we see the aforementioned 
asymmetry as the lengths of the result pointer lists are each a function of 
how many other instruction sequences will use each result. In contrast to the 
result centric tableau approach, this information is not a property of the 
given instruction sequence, rather it is a property of the data flow graph the 
sequence occurs in. 


We have an analogous hazard to that of the result centric approach. Suppose 
that a given instruction sequence just ran and its output is to be used by a 
second downstream instruction sequence, but not on its next invocation, but 
rather the one after that. Again, among the potential fixes we could use a 
different link for the second invocation, or make one of the inputs to the 
downstream instruction stream a FIFO. 


State variables and messages 


We may allocate variables on a tableau which are interpreted as the state of 
the program rather than as input or output of instruction sequences. 
Similarly, we can keep messages. The FIFOs suggested above are examples of 
one possible messaging scheme. 


Hardware emulation 


Suppose we have a special situation where we have a collection of instruction 
sequences where no member makes a direct execution call to any other, but 
rather each instruction sequence runs when its inputs change. Now suppose 
also, that we have a rule that an instruction sequence should run if one of its 
inputs changes. 


We modify the link to look something like this: 


typedef struct{ 
SQ:Ptr sequence; 
SQ:Args *arg_froms; 
SQ:Args *arg_block; 
SQ:Ress *res_ block; 
. } SQ:Lnk; 


Dro RIOS E 


As usual we have a pointer to the instruction sequence being called at the top 
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of the structure. As per the result centric approach, line 3 arg_froms, is a 
list of pointers telling us where to get the input arguments from. As per the 
input centric approach, Line 4 arg_block, is a pointer to a block on the 
tableaux for the inputs. Again like for the result centric approach, Line 5 
res_ block, is a pointer to a block of results. There is no list of continuations, 
as control flow is determined by inputs changing rather than direct 
invocation. 


To run our program, we repeat the following steps until no inputs change: 


1. For each and every link, use the args_from to write the arg_block 
with input values. For each link, if any of the arg_block value is 
changed by the write, put a pointer to it on the execution queue. 


2. Ifthe execution queue is empty, then take the finished continuation. 
Otherwise, pull all the links off the instruction queue and run their 
instruction sequences. This will populate each res_block; continue 
to step 1. 


Notice that in step 2 that all of the instruction sequences are independent of 
each other. Hence if our tableau is shared between threads, then we can 
spawn many threads, up to the number of instruction sequences, so as to get 
the work done as soon as possible. Indeed, in place of an instruction 
sequence, there could be a hardware processor instead. 


We may organize any feed forward data flow graph in levels, with inputs 
going into level 0, and results provided from level n. We add identity function 
instruction sequences into the levelized graph so that each instruction 
sequence on level n receives inputs only from instruction sequences on level 
n-1. We have an instruction queue for each level. 


With the approach where instruction sequences are scheduled to run 
whenever an input changes, execution time can be wasted on incomplete 
input sets, where one input has arrived, but another has not. This is a typical 
data flow issue. If we do not want the spurious firings, we must somehow 
recognize matched inputs sets. The data flow solution is to add tags to data 
and then to match up the tags. In superscalars we rename registers and 
instructions call out registers as operands. With a levelized data flow graph 
data remains in sync as it passes from level to level. 


The levelized approach also facilitates pipelined execution, by having 
different execution queues on each level. Instruction sequences on each level 
specific queue would be working on synchronized input data, and would be 
independent of each other. Thus we could put each execution queue on a 
separate thread, and then have those threads rendezvous, after which we 
would propagate data, and repeat. After the initial pipeline flush we would 
produce results upon each iteration. 


This approach works for emulating hardware. For example, our instruction 
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sequences could be used to emulating logic gates in a logic circuit. 


Note data driven execution has risk of performing a lot of superfluous work. 
Consider for example a conventional program, where there are different 
continuations called on each branch of the conditional. With data driven 
execution both of those continuations would potentially be watching the 
same inputs and upon a change both be scheduled for execution, even though 
only one will produces results that are of use to us. 


We might try to fix this problem by introducing the equivalent of chip select, 
i.e. a special input that determines if an instruction sequence should be 
placed in the execution queue. In which case we have come full circle, as this 
is essentially what is done in the conventional execution model when an 
instruction is called (or not called). 


Even with explicit calls we could keep the execution queue. Accordingly 
instead of a call dispatching an instruction sequence for immediate 
execution, it would put a pointer to it on the queue. The queue then turns 
into a point of control for dispatch, and thus would enable pre-emption at 
the granularity of an instruction sequence, scheduling, and prioritization. 


Latent Linking Example 


It will often be the case that the implementation of a general use sequences 
will make use of further general use sequences. For sake of discussion, 
suppose that we have five sequences, s0, s1, s2, s3, and s4. Further suppose 
that s0 is written by an application programmer. The application 
programmer makes use of the general use library sequence s1. The 
application programmer has given s1 two continuations, say for sake of 
discussion they are called nominal and fail. The application programmer 
probably does not really care how s1 is implemented, and in fact that 
implementation might change with new versions of the library. 


However, s1 does have an implementation. Say for sake of example that s1 
will declare its own result tableau to be used for sequences inside of it, and 
thus that it will wire up the links between the sequences much like our top 
level code does. Hence, s1 might wire together s2, s3, and s4 and then call s4. 


At run time s0 will call s1. s1 will then call, say, s2, which calls s3, then s4. 
However, the application programmer only sees s1, and he or she expects 
that s1 will take one of the provided continuations, either nominal or fail. 
But after s1 calls s2, control flow will never go back to s1. Suppose for 
example the wave of execution reaches s4 when it is time to call nominal or 
fail. 


If s4 is a special use instruction sequence, then we can write s4 to use the 
continuation links originally passed to s1. If s4 is a general use instruction 
sequence, then it will have its own link, but that link will not know anything 
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about nominal and fail. Because those were not available until s1 was called. 


What we need to have happen is for s1 to run a latent linker to connect s4 to 
continuations to the nominal or fail continuations. Accordingly upon entry 
to s1, before it continues to s2, s1 copies continuations passed to it to where 
they belong in the circuit of s2, s3, and s4. 


Hence, part of the job of a general instruction sequence that itself makes use 
of general instruction sequences is to finish building the data flow graph, i.e. 
to perform a latent link, before it continues into it first constituent sub- 
sequence. Thus, we have a sort of fractal pattern where every invocation of a 
general use instruction sequence has some differential work to be done of 
the same sort that was done when wiring the outer instruction sequences 
together. 


Here is an example from the TM2x tape implementation. This routine copies 
elements from one tape machine to another. It does this by scaling the passed 
in element extent to byte extents, and then calling copy_bytes. 
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SQ:def(TM2x:copy_elements) { 
TM2x:CopyElements:Lnk *Ink = (TM2x:CopyElements:Lnk *)SQ:Ink; 


// tableau 

address_t src_byte_0; 

address_t dst_byte_0; 

address_t ext_byte_n; 

SQ:make_Lnk(scale_src ,Inclusive:30pLL ,&&Inclusive:mul_idx); 
SQ:make_Lnk(scale_dst ,Inclusive:-30pLL ,&&Inclusive:mul_idx); 
SQ:make_Lnk(scale_ext ,Inclusive:-30pLL ,&&Inclusive:mul_ext); 
SQ:make_Lnk(copy_bytes ,TM2x-CopyBytes ,&&TM2x:copy_bytes); 


scale_src_Inks = (Inclusive:-30pLL-Lnks) 
{ .nominal = AS(scale_dst_Ink ,SQ:Lnk) 
,-gt_address_t_n = Ink->Inks->src_index_gt_n 


hi 
scale_dst_Inks = (Inclusive:-30pLL-Lnks) 
{ .nominal = AS(scale_ext_Ink ,SQ-Lnk) 
,.gt_address_t_n = Ink->Inks->dst_index_gt_n 


scale_ext_Inks = (Inclusive:30pLL:Lnks) 
{ .nominal = AS(copy_bytes_Ink ,SQ:Lnk) 
,.gt_address_t_n = Ink->Inks->src_index_gt_n 


Fi 

copy_bytes_Inks = (TM2x:CopyBytes:Lnks) 

{ .nominal = Ink->Inks->nominal 
,-src_index_gt_n = Ink->Inks->src_index_gt_n 

,-dst_index_gt_n = Ink->Inks->dst_index_gt_n 


ti 


scale_src_ress.r = &src_byte_0; 
scale_dst_ress.r = &dst_byte_0; 
scale_ext_ress.r = &ext_byte_n; 


scale_src_args = (Inclusive:30pLL-Args) 
{ .a_0 = Ink->args->src_element_0O 
„a_i = Ink->args->element_byte_n 


scale_dst_args = (Inclusive-30pLL-Args) 
{ .a_0 = Ink->args->dst_element_0O 
„a_i = Ink->args->element_byte_n 
F 
scale_ext_args = (Inclusive:30opLL-Args) 
{ .a_O = Ink->args->element_n 
„a_i = Ink->args->element_byte_n 


y 
copy_bytes_args = (TM2x:CopyBytes:Args) 
{ .src = Ink->args->src 
,src_byte_0O = &src_byte_0 
„dst = Ink->args->dst 
,dst_byte_O = &dst_byte_0 
,byte_n = &ext_byte_n 
y 


SQ-continue_indirect(scale_src_lnk); 


} SQ'end(TM2x:copy_elements); 
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The only macro that has not yet been discussed thus far is AS. AS is a strong, 
no warnings given, cast. The above code uses the same format as we have 
been following up to this point. First it declares a result centric tableau. Then 
it declares the topology. After creating the links, it wires up the connections 
between the sequences. Notice that during this wiring up the sequence step, 
it copies-down links from SQ:Ink (which has been cast to a more specific 
type and is now just called Ink) It then ties the arguments and results of the 
local sequences into the local result centric tableau. 


Clean array iteration 


Suppose our machine tape is an indexed array, and we want to be maximally 
efficient in representing indexes on to the tape, and we do not want to force 
the tape to expand if not necessary. 


In general array indexes run from zero to n, and thus our array has n+1 cells 
for holding data. We call n the extent of the array. We call n+1 the length of 
the array. 


O index of the leftmost cell 
n extent of the array 
n+1 length of the array 


In most all computer architectures today addresses logically locate bytes in 
memory. I say logically because various processor realizations might pull 
larger units from memory, and then select a byte as a separate step, but this 
does not change the logical result. 


At the time of this writing the industry standard length of a byte is 8 bits, but 
if extra bits were added to the byte tomorrow, most programs would not 
notice it. This is because bytes are addressed and atomically copied. If we 
want a specific bit within a byte, we almost always have to use special 
instructions to get it. 


Data that has primitive type may be block copied using a function such as 
memcpy. In other words, such data is a byte array. An IEEE std. 754 floating 
point number. is an example. In data that has complex type will have internal 
pointer structures that, in general, we must be aware of if we want to copy 
them. Examples include linked list and tree containers. This is not the whole 
story about copying, but it is description enough for our immediate needs. 


It follows that all arrays of elements with primitive type, are also arrays of 
bytes. This gives us at least two different units we may use for measuring the 
length of an array, where one is the number of bytes, and the other is the 
number of elements. We have three interest extent numbers. That of the 
maximum element index, that of the maximum byte index, and that of the 
index of the first byte of the element with the maximum index. 
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It is even possible that we might have more than one element type to 
consider for a given array. It follows that in a context where it is not clear we 
need to state the data type being used by saying the ‘length in uint32_t’ or 
‘length in bytes’. In C argo the length in bytes is called the size, but I prefer 
the more general terminology that specifies the element type. In C, the length 
in bytes, i.e. the size, is used in malloc calls, while the length of the in bytes 
for the data type bound to a pointer is automatically added to a pointer when 
it is increment. In conventional C programming a loop bound will be 
specified as the length of an array as measured in number of elements. 


For sake of discussion, suppose that we have some sort of simple controller, 
and our fundamental type is a byte, even for addressing. Or similarly that we 
desire for our memory access to remain on a single page of virtual memory, 
where pages are 256 bytes, and thus the page offset is a single byte. Then our 
memory, or our page, is an array of length 256 = 2° bytes. 


Now suppose we desire to iterate through this area of memory. An index into 
the array, i.e. an address to a cell on our tape, will fit in 8 bits, i.e. one byte. 
With such an index we may address any cell, including the one with the 
maximum index, cell index 255. Yet, it is conventional in computing to use 
the length of an array as an exclusive bound in loops. For example, we might 
have something like: 


i=0; 

n=255; 
while( i < n+1 ){ 
printf(“i: %u\n” i); 


itt; 


} 


When using a one byte integer the bounding value n+1 will overflow, and 
typically wrap back to zero. In this case the loop would not be entered. 


If we expand the representation of the bounding value to n+ 1 to two bytes, 
then we can represent 256, and thus we will enter the loop. This is wasteful 
as about half the bits are not used, but at least we can represent this outer 
exclusive bound. 


Instead, we could try to fix the problem by changing the loop test so that the 
bounding value may be n, it would look like this: 
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i=0; 

n=255; 

while( i <= n ){ 
printf(“i: You\n” i); 


Itt; 


} 


However now there is a different problem. On what is to be the last pass 
through the loop we will increment i from 255 to 256, but i is one byte, and 
this will typically wrap i back to zero. Zero is less than 256 so, far from fixing 
the problem, we have made it worse. This loop will run forever. 


The only reason that we have these end cases is because we think in terms 
of exclusive bounds. Instead of saying "tell me if you cannot stay on the tape", 
we are saying "tell me if you go off the tape". 


In addition to potentially causing program errors, we have also potentially 
created some hazards for enhanced performance hardware. When a 
processor tries to increase performance by watching a pointer and pre- 
fetching data when an address value changes, it might walk off the current 
page resulting in an unnecessary data fetch and possibly a page miss. If the 
next page does not exist, we will have to distinguish between page faults 
caused by prefetches from real ones caused by program behavior. If we 
return some data anyway, we might have security issues. It is very interesting 
that theoretical desire not to expand the tape and to use efficient indexing is 
so directly related to performance and security. 


loop: { 
printf(“i: Yu\n” ,i); 


if( i < 255 ){ 
ety 
goto loop; 
J 
} 


If we instead use inclusive bounds, we do not have these end cases. Because 
the bound is in the area that we are iterating over, it will be representable in 
the same data type as an index, and any speculative execution and prefetch 
based on watching register values will be for data that is part of the iteration 
work. 


146/148 


C does not have a built in inclusive bound loop structure, so here we 
construct it using a label and a goto. Note the entry into the loop is not 
guarded. This goes back to our discussion on levels of computation and 
emptiness. Here we are working with first level areas. The smallest array 
occurs when the leftmost cell index is zero, and the extent is also zero, i.e. 
when the area has one cell. We must add a second level structure to support 
the concept of the array being empty. 


if( empty ){ 
printf(“This empty array has no index values.\n”); 
yelse{ 
i=0; 
n=255; 


loop:{ 
printf(“i: Yu\n” ,i); 


fi 1 <= 255) 
[arare 
goto loop; 
J 
Y 


There is no reason that a C compiler cannot compile for loops or related 
while loops into their inclusive bound counterparts. For the TTCA we will 
use extent values and inclusive bounding instead of length values and 
exclusive bounding. 


Sometimes we desire to change the units the extent is measured in. Say for 
example we have an array with an extent of 31 as measured against 32 bit 
integers. Now we would like to know the extent of the array in bytes. The 
extent of a 32 bit integer is 3 bytes. We may use this information along with 
a special multiple instruction to find the answer: 


mul_ext(3 , 31) =3*31+3+31=127 


Here mul_extis intended to be a single instruction in our new instruction set. 
It computes a0 * al + a0 + al as a single instruction. 


Another useful multiply variation is used for converting an index in one unit 
of measure into an index in another unit of measure. It computes a0 * al + 
a0 as a single instruction. Take for example that we have an array of 32 bit 
integers. The cell that we are indexing is found at index 4. Now we know that 
we have little endian packing and desire to have a byte index to the least 
significant byte of that same integer. 
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mul_idx(4,3)=4*3+4=16 


Thus given an array of integers, array[4] produces an integer, say i. For that 
same array now indexed in bytes, array[16] will be the least significant byte 
of i. 


Tableau Optimizations 


Allocations on the tableau have lifetimes. For sake of example, say the given 
lifetime of an allocation spans from t0 to t7 inclusively, and the lifetime of 
another allocation spans from t11 to t21. Then the second allocation can use 
the same memory as that used by the first allocation. A couple of examples 
follow. 


Consider single threaded execution with a result centric tableau. Only one 
link can be active at any given time. Suppose the active link is 1.0. Now 
examine the input pointers for all downstream links from 1_0. These can be 
found by traversing the continuations. If none of these downstream links 
have inputs that point to an upstream link's result block, then the upstream 
result block can be deallocated. Given a data flow diagram we can perform 
this analysis on a per node basis at compile time. 


As another example of optimization, consider the case of single threaded 
execution over a levelized data flow diagram. When we are at any given level, 
say level n, then we know that allocations associated with all prior levels can 
be released. 


Streaming 


One option for streaming is to use stack allocated tableaux and then to spawn 
a new thread for each new input data set. If we want to keep results in order, 
we will have to keep track of the order that the threads were spawned in, and 
to collect the output data in that order. 


Another option for streaming was described earlier, and that is to create a 
levelized data flow graph of our code, and then to run data through as 
pipeline. 
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